diff --git a/docs/data/generated/dev-docs.json b/docs/data/generated/dev-docs.json index 5679109e8..4013e0162 100644 --- a/docs/data/generated/dev-docs.json +++ b/docs/data/generated/dev-docs.json @@ -412,7 +412,7 @@ "url": "/dev/prds/plugin-settings-live/", "title": "Plugin settings reconcile live", "section": "PRDs", - "body": "# Plugin settings reconcile live\n\n**Status:** Draft\n**Tracking:** GitHub [#1154](https://github.com/samcharles93/archie-core/issues/1154), epic [#998](https://github.com/samcharles93/archie-core/issues/998)\n**Authority:** `docs/prds/runtime-control-plane.md` decides each feature owns how\nits changes take effect; `docs/architecture/plugins-and-extensions.md` owns the\nplugin engine rule; `docs/prds/control-plane-apply-status.md` owns where apply\nreports go.\n\n## Outcome\n\nAn operator drops a plugin, module or secret-engine file into the configured\ndirectory while archied runs, and the daemon loads it without a restart. An\nedit to an installed file replaces what is running. Deleting an installed file\nnever stops running code; apply status says so. A stored change to\n`plugin-settings` applies live for additions instead of requiring a restart.\n\n## Triggers\n\n`plugin-settings` joins the kinds archied watches (`PluginSettingsKind` moves\nto the live list in `internal/app/archied/control_plane.go`). After the daemon\nlayers a new version it reconciles the three directories the document names.\nA second trigger needs no document change: a poller reconciles on an interval,\nso a file dropped into a directory loads on its own. The poll interval is the\napply-status restamp constant; no filesystem-watch dependency is added. The\npoller reads the current document's directories from the running config each\ntick, so a changed `plugin_dir`, `module_dir` or `secret_engine_dir` retargets\nreconciliation without a restart.\n\n## Rules\n\n- Adding and changing load; removing never unloads. Yaegi interpreters are not\n reclaimed while a caller still exists and there is no unload path; the\n running set can only grow.\n- Each file is loaded through a fresh interpreter (the boot pattern). A changed\n file therefore loads cleanly, and replacement switches consumers to the new\n interpretation; files are content-hashed so only new and changed files load.\n- A file that fails to load is skipped and reported; already-running code keeps\n running. This is the degrade-and-skip convention `plugin.LoadDir` and\n `Registry.LoadDir` already apply at boot. Boot keeps its reject-at-load rule\n for the module directory; a broken file at runtime skips without aborting.\n- A directory that does not exist reconciles to an empty set.\n\n## The three registries\n\n- **Secret engines** (`internal/secret`). Reconciliation re-runs\n `Registry.LoadDir` against the live directory. `Register` replaces by name,\n so an edited engine takes effect for the next resolution. A deleted engine\n stays registered: existing references keep resolving through it until\n restart, and the record says so. Engines are stateless resolvers, so a swap\n needs no lifecycle step.\n- **Modules** (`internal/domain/eda/module`): reconciliation re-runs\n `ModuleRegistry.Register` for the kind's file. The invoker maps\n last-write-wins, and playbook dispatch resolves through the registry at\n invoke time, so the swap reaches a running playbook store unchanged. A file\n deleted for a kind leaves the old invoker in place, running; reported.\n- **Plugins** (`internal/plugin`): `Host` grows running-state operations beside\n `Register`, `Start` and `Stop`. One adds a single module while the host is\n running: it is inserted in dependency order, started alone, and on failure\n rolled back alone, leaving every previously started module running. The\n other replaces a module whose manifest id matches: stop it, start the new\n one, and restart the old one when the new start fails. Replacing a module\n that another depends on is refused, not swapped. These operations obey the\n plugin engine rule: the host still invokes start and stop itself, isolates\n the failure to the one module, and keeps a rollback. A deleted file makes no\n host call: the module runs on and is reported.\n\n## `skills_dir`\n\n`skills_dir` is not part of this change. The daemon refuses a candidate\n`plugin-settings` document whose `skills_dir` differs from the value it is\nrunning, reporting that the field applies on restart, until a skill\nreconciliation exists to carry it. The refusal is the existing rejected-edit\napply-status shape: the store keeps the version, the document is not applied,\nlast-known-good keeps running.\n\n## Apply status\n\nOne record: `archied` on `plugin-settings`. The reconciliation outcome is\nreported against the version the process is running. The record's error field\ncarries the outstanding reconciliation problem: a deleted file still running, a\nfile that failed to load, or the directory failing to read. A reconciliation\nwith nothing outstanding clears the error. The version stays the version the\nprocess is running; an add that succeeded is live even while a removal is\noutstanding, and the error says so rather than showing the kind as\nrestart-required. archie-gateway keeps boot-time loading; its record simply\nstays on the version it last applied until it restarts, which the per-process\napply-status page already reads correctly." + "body": "# Plugin settings reconcile live\n\n**Status:** Draft\n**Revision:** 2\n**Tracking:** GitHub [#1154](https://github.com/samcharles93/archie-core/issues/1154), epic [#998](https://github.com/samcharles93/archie-core/issues/998)\n**Authority:** `docs/prds/runtime-control-plane.md` decides each feature owns how\nits changes take effect; `docs/architecture/plugins-and-extensions.md` owns the\nplugin engine rule; `docs/prds/control-plane-apply-status.md` owns where apply\nreports go.\n\n## Outcome\n\nAn operator drops a plugin, module or secret-engine file into a configured\ndirectory while archied runs, and the daemon loads it without a restart. An\nedit to an installed file replaces what is running. A delete never stops running\ncode; apply status says so. A stored `plugin-settings` change applies live for\n`plugin_dir`, `module_dir` and `secret_engine_dir`; a change to `skills_dir` is\nrefused.\n\n## Triggers\n\n`PluginSettingsKind` joins `runtimeResourceKinds` in\n`internal/app/archied/control_plane.go`, and its descriptor `ApplyMode` becomes\n`live` in `internal/app/controlplane/operational_settings.go`.\n\nA poller reconciles on the apply-status restamp interval, so a dropped file\nloads without a filesystem watch. Each tick re-runs both halves the boot path\nruns, against the directories in the running config: the directory loads\n(`secret.Registry.LoadDir`, `ModuleRegistry.Register`, `plugin.LoadDir`) and the\nlayering (`boot.runtimeConfig`), which re-resolves provider credentials. A\nprovider disabled at boot for an unresolvable engine is re-resolved on the first\ntick that finds it; a changed directory retargets the next tick without a\nrestart.\n\n## Rules\n\n- Adding and changing load; removing never unloads. Yaegi interpreters are not\n reclaimed while a caller still exists and there is no unload path; the running\n set can only grow. A removal stays outstanding in the apply-status error until\n the process restarts.\n- Each file is loaded through a fresh interpreter (the boot pattern) and\n content-hashed, so only new and changed files load.\n- A file that fails to load is skipped and reported; already-running code keeps\n running, the degrade-and-skip convention `plugin.LoadDir` and\n `Registry.LoadDir` already apply at boot. Boot keeps its reject-at-load rule\n for the module directory; a broken file at runtime skips without aborting.\n- A directory that does not exist reconciles to an empty set.\n\n## The three registries\n\n- **Secret engines** (`internal/secret`). Reconciliation re-runs\n `Registry.LoadDir` against the live directory. `Register` replaces by name, so\n an edited engine takes effect for the next resolution and a dropped engine\n resolves on the next tick. A deleted engine stays registered: existing\n references keep resolving through it until restart. Engines are stateless\n resolvers, so a swap needs no lifecycle step.\n- **Modules** (`internal/domain/eda/module`). Reconciliation re-runs\n `ModuleRegistry.Register` for the kind's file. Playbook dispatch resolves\n through the registry at invoke time, so the swap reaches a running playbook\n store unchanged; a deleted file leaves the old invoker in place, running.\n- **Plugins** (`internal/plugin`). `Host` grows running-state operations beside\n `Register`, `Start` and `Stop`, and `Register` keeps its rule that\n registration is allowed only before the first `Start`, so these are new host\n methods rather than a relaxed `Register`. One adds a single module while the\n host is running: inserted in dependency order, started alone, and rolled back\n alone on failure, leaving every previously started module running. The other\n replaces a module whose manifest id matches: stop it, start the new one, and\n restart the old one when the new start fails; replacing a module another\n running module depends on is refused. A module's `Start` is its only check\n (`plugin.Module` has no probe, and one manifest id cannot be registered\n twice), so a replacement checks by starting and keeps a rollback rather than\n switching first. These operations obey the plugin engine rule: the host\n invokes start and stop itself and isolates the failure to one module. A\n deleted file makes no host call: the module runs on.\n\n## `skills_dir`\n\n`skills_dir` is not part of this change. The daemon refuses a `plugin-settings`\ndocument whose `skills_dir` differs from the value in force. The check is a\npre-layer comparison on the live path (`boot.applyRuntimeResourceUpdate` in\n`internal/app/archied/control_plane.go`), not an addition to\n`validatePluginSettings` in `internal/app/controlplane/operational_settings.go`,\nwhich stays value-agnostic per the #1143 decision recorded there: a\nchange-immutability rule needs the running config, and #1143's validates-nothing\nstill holds for values. The write lands, the store keeps the version, the daemon\npublishes nothing, the refusal is reported against that stored version, and\nlast-known-good keeps running.\n\nThe layering assigns all four directories from one document, so a refused\n`skills_dir` refuses the whole document and the other kinds still layer. That is\nacceptable: `skills_dir` is the one field with no live consumer, a\nwhole-document refusal is the honest signal that none of it took effect, and an\noperator changing `skills_dir` alongside another directory writes the two\nchanges separately.\n\n## Apply status\n\nOne record per process: `archied` on `plugin-settings`, reported against the\nversion the process is running. The record's error field carries the outstanding\nproblem: a deleted file still running, a file that failed to load, the directory\nfailing to read, or a refused replace of a module another running module depends\non. A clean reconciliation clears the error. An add that succeeded is live even\nwhile a removal is outstanding, and the error says so rather than showing the\nkind as restart-required.\n\n`ApplyMode` becomes `live`, so `RestartPendingBanner` no longer names\n`plugin-settings`. The gateway's lag is shown instead by the per-process\napplied-version rows on the Plugins page, which read the state\n`applyStatusForKind` derives from `applied_version \u003c storedVersion` in\n`ui/src/stores/control-plane.ts`; that derivation is independent of\n`ApplyMode`.\n\n`archie-gateway` layers the `plugin-settings` document at boot through\n`boot.loadRuntimeConfig` and reports it, but never calls `boot.loadPlugins` or\n`boot.loadModules` and never reads the four directories, so it loads no plugin,\nmodule or secret engine. With no live watch its row lags the stored version\nuntil restart; that lag is a reporting lag, not a behaviour gap.\n\n## Verification\n\n- A plugin, module and secret-engine file dropped while the daemon runs loads on\n the next tick; an unchanged file is not reloaded.\n- An edited file replaces the running interpretation; a deleted file leaves the\n running code running, with the process's error naming it until restart.\n- A `skills_dir` change refuses the document: the store keeps the version, the\n record carries that version and the refusal, and the running `skills_dir` is\n unchanged.\n- A plugin replacement whose new `Start` fails restarts the old module and\n reports the failure; a replacement of a depended-on module is refused and the\n dependent keeps running.\n- A provider disabled at boot for a missing engine resolves again after the\n engine file loads and a tick re-layers.\n- The gateway reports the last version it layered at boot, loads no directory,\n and its lag appears in its applied-version row rather than the restart banner.\n" }, { "url": "/dev/prds/pr-review-agent/", diff --git a/docs/prds/plugin-settings-live.md b/docs/prds/plugin-settings-live.md index 07644e398..a7e42b082 100644 --- a/docs/prds/plugin-settings-live.md +++ b/docs/prds/plugin-settings-live.md @@ -1,6 +1,7 @@ # Plugin settings reconcile live **Status:** Draft +**Revision:** 2 **Tracking:** GitHub [#1154](https://github.com/samcharles93/archie-core/issues/1154), epic [#998](https://github.com/samcharles93/archie-core/issues/998) **Authority:** `docs/prds/runtime-control-plane.md` decides each feature owns how its changes take effect; `docs/architecture/plugins-and-extensions.md` owns the @@ -9,34 +10,38 @@ reports go. ## Outcome -An operator drops a plugin, module or secret-engine file into the configured +An operator drops a plugin, module or secret-engine file into a configured directory while archied runs, and the daemon loads it without a restart. An -edit to an installed file replaces what is running. Deleting an installed file -never stops running code; apply status says so. A stored change to -`plugin-settings` applies live for additions instead of requiring a restart. +edit to an installed file replaces what is running. A delete never stops running +code; apply status says so. A stored `plugin-settings` change applies live for +`plugin_dir`, `module_dir` and `secret_engine_dir`; a change to `skills_dir` is +refused. ## Triggers -`plugin-settings` joins the kinds archied watches (`PluginSettingsKind` moves -to the live list in `internal/app/archied/control_plane.go`). After the daemon -layers a new version it reconciles the three directories the document names. -A second trigger needs no document change: a poller reconciles on an interval, -so a file dropped into a directory loads on its own. The poll interval is the -apply-status restamp constant; no filesystem-watch dependency is added. The -poller reads the current document's directories from the running config each -tick, so a changed `plugin_dir`, `module_dir` or `secret_engine_dir` retargets -reconciliation without a restart. +`PluginSettingsKind` joins `runtimeResourceKinds` in +`internal/app/archied/control_plane.go`, and its descriptor `ApplyMode` becomes +`live` in `internal/app/controlplane/operational_settings.go`. + +A poller reconciles on the apply-status restamp interval, so a dropped file +loads without a filesystem watch. Each tick re-runs both halves the boot path +runs, against the directories in the running config: the directory loads +(`secret.Registry.LoadDir`, `ModuleRegistry.Register`, `plugin.LoadDir`) and the +layering (`boot.runtimeConfig`), which re-resolves provider credentials. A +provider disabled at boot for an unresolvable engine is re-resolved on the first +tick that finds it; a changed directory retargets the next tick without a +restart. ## Rules - Adding and changing load; removing never unloads. Yaegi interpreters are not - reclaimed while a caller still exists and there is no unload path; the - running set can only grow. -- Each file is loaded through a fresh interpreter (the boot pattern). A changed - file therefore loads cleanly, and replacement switches consumers to the new - interpretation; files are content-hashed so only new and changed files load. + reclaimed while a caller still exists and there is no unload path; the running + set can only grow. A removal stays outstanding in the apply-status error until + the process restarts. +- Each file is loaded through a fresh interpreter (the boot pattern) and + content-hashed, so only new and changed files load. - A file that fails to load is skipped and reported; already-running code keeps - running. This is the degrade-and-skip convention `plugin.LoadDir` and + running, the degrade-and-skip convention `plugin.LoadDir` and `Registry.LoadDir` already apply at boot. Boot keeps its reject-at-load rule for the module directory; a broken file at runtime skips without aborting. - A directory that does not exist reconciles to an empty set. @@ -44,45 +49,86 @@ reconciliation without a restart. ## The three registries - **Secret engines** (`internal/secret`). Reconciliation re-runs - `Registry.LoadDir` against the live directory. `Register` replaces by name, - so an edited engine takes effect for the next resolution. A deleted engine - stays registered: existing references keep resolving through it until - restart, and the record says so. Engines are stateless resolvers, so a swap - needs no lifecycle step. -- **Modules** (`internal/domain/eda/module`): reconciliation re-runs - `ModuleRegistry.Register` for the kind's file. The invoker maps - last-write-wins, and playbook dispatch resolves through the registry at - invoke time, so the swap reaches a running playbook store unchanged. A file - deleted for a kind leaves the old invoker in place, running; reported. -- **Plugins** (`internal/plugin`): `Host` grows running-state operations beside - `Register`, `Start` and `Stop`. One adds a single module while the host is - running: it is inserted in dependency order, started alone, and on failure - rolled back alone, leaving every previously started module running. The - other replaces a module whose manifest id matches: stop it, start the new - one, and restart the old one when the new start fails. Replacing a module - that another depends on is refused, not swapped. These operations obey the - plugin engine rule: the host still invokes start and stop itself, isolates - the failure to the one module, and keeps a rollback. A deleted file makes no - host call: the module runs on and is reported. + `Registry.LoadDir` against the live directory. `Register` replaces by name, so + an edited engine takes effect for the next resolution and a dropped engine + resolves on the next tick. A deleted engine stays registered: existing + references keep resolving through it until restart. Engines are stateless + resolvers, so a swap needs no lifecycle step. +- **Modules** (`internal/domain/eda/module`). Reconciliation re-runs + `ModuleRegistry.Register` for the kind's file. Playbook dispatch resolves + through the registry at invoke time, so the swap reaches a running playbook + store unchanged; a deleted file leaves the old invoker in place, running. +- **Plugins** (`internal/plugin`). `Host` grows running-state operations beside + `Register`, `Start` and `Stop`, and `Register` keeps its rule that + registration is allowed only before the first `Start`, so these are new host + methods rather than a relaxed `Register`. One adds a single module while the + host is running: inserted in dependency order, started alone, and rolled back + alone on failure, leaving every previously started module running. The other + replaces a module whose manifest id matches: stop it, start the new one, and + restart the old one when the new start fails; replacing a module another + running module depends on is refused. A module's `Start` is its only check + (`plugin.Module` has no probe, and one manifest id cannot be registered + twice), so a replacement checks by starting and keeps a rollback rather than + switching first. These operations obey the plugin engine rule: the host + invokes start and stop itself and isolates the failure to one module. A + deleted file makes no host call: the module runs on. ## `skills_dir` -`skills_dir` is not part of this change. The daemon refuses a candidate -`plugin-settings` document whose `skills_dir` differs from the value it is -running, reporting that the field applies on restart, until a skill -reconciliation exists to carry it. The refusal is the existing rejected-edit -apply-status shape: the store keeps the version, the document is not applied, +`skills_dir` is not part of this change. The daemon refuses a `plugin-settings` +document whose `skills_dir` differs from the value in force. The check is a +pre-layer comparison on the live path (`boot.applyRuntimeResourceUpdate` in +`internal/app/archied/control_plane.go`), not an addition to +`validatePluginSettings` in `internal/app/controlplane/operational_settings.go`, +which stays value-agnostic per the #1143 decision recorded there: a +change-immutability rule needs the running config, and #1143's validates-nothing +still holds for values. The write lands, the store keeps the version, the daemon +publishes nothing, the refusal is reported against that stored version, and last-known-good keeps running. +The layering assigns all four directories from one document, so a refused +`skills_dir` refuses the whole document and the other kinds still layer. That is +acceptable: `skills_dir` is the one field with no live consumer, a +whole-document refusal is the honest signal that none of it took effect, and an +operator changing `skills_dir` alongside another directory writes the two +changes separately. + ## Apply status -One record: `archied` on `plugin-settings`. The reconciliation outcome is -reported against the version the process is running. The record's error field -carries the outstanding reconciliation problem: a deleted file still running, a -file that failed to load, or the directory failing to read. A reconciliation -with nothing outstanding clears the error. The version stays the version the -process is running; an add that succeeded is live even while a removal is -outstanding, and the error says so rather than showing the kind as -restart-required. archie-gateway keeps boot-time loading; its record simply -stays on the version it last applied until it restarts, which the per-process -apply-status page already reads correctly. \ No newline at end of file +One record per process: `archied` on `plugin-settings`, reported against the +version the process is running. The record's error field carries the outstanding +problem: a deleted file still running, a file that failed to load, the directory +failing to read, or a refused replace of a module another running module depends +on. A clean reconciliation clears the error. An add that succeeded is live even +while a removal is outstanding, and the error says so rather than showing the +kind as restart-required. + +`ApplyMode` becomes `live`, so `RestartPendingBanner` no longer names +`plugin-settings`. The gateway's lag is shown instead by the per-process +applied-version rows on the Plugins page, which read the state +`applyStatusForKind` derives from `applied_version < storedVersion` in +`ui/src/stores/control-plane.ts`; that derivation is independent of +`ApplyMode`. + +`archie-gateway` layers the `plugin-settings` document at boot through +`boot.loadRuntimeConfig` and reports it, but never calls `boot.loadPlugins` or +`boot.loadModules` and never reads the four directories, so it loads no plugin, +module or secret engine. With no live watch its row lags the stored version +until restart; that lag is a reporting lag, not a behaviour gap. + +## Verification + +- A plugin, module and secret-engine file dropped while the daemon runs loads on + the next tick; an unchanged file is not reloaded. +- An edited file replaces the running interpretation; a deleted file leaves the + running code running, with the process's error naming it until restart. +- A `skills_dir` change refuses the document: the store keeps the version, the + record carries that version and the refusal, and the running `skills_dir` is + unchanged. +- A plugin replacement whose new `Start` fails restarts the old module and + reports the failure; a replacement of a depended-on module is refused and the + dependent keeps running. +- A provider disabled at boot for a missing engine resolves again after the + engine file loads and a tick re-layers. +- The gateway reports the last version it layered at boot, loads no directory, + and its lag appears in its applied-version row rather than the restart banner.