feat(messaging): restart only the channel whose settings changed
archie-core-zfb0.4. Channel settings were restart-required as a whole: a
change to one channel needed a process restart that restarted every channel.
The Messaging Service now reconciles the stored channel-settings resource on
the apply-status restamp interval (docs/prds/plugin-settings-live.md), re-layers
it over the file document exactly as boot does, and restarts only the channel
whose transport settings changed. Every other channel keeps serving.
A restart is stop-and-start, not an in-place reload: each channel gets its own
supervisor and a rebuild factory, so email and webhook -- which have no reload
seam -- are restarted too. Telegram's Start still calls dropPendingUpdates
before it long-polls, so the rule holds on the reconcile path as on /restart. A
replacement is built before the running channel is touched, so a configuration
this build cannot run leaves the running channel serving.
The composed set and the bound listen addresses are process bindings: enabling
or disabling a channel, or changing chat.webhook_addr / chat.email.listen_addr,
is refused and reported for a restart rather than applied. A channel rebuild
that fails is not recorded as applied, so it is retried on the next tick. The
applied version reaches apply status, and ChannelSettingsKind flips to live.
Files: internal/app/archiemessaging/{compose,reconcile,supervise,run}.go,
controlplane ApplyMode, archied runtime watch, configuration.md, the config
skill matrix and the regenerated dev-docs artifact.
Verified: go test ./internal/app/archiemessaging/... (race clean); go test
./cmd/archie-messaging/... ./internal/app/controlplane/... ./internal/app/archied/...
./internal/config/...; go vet and golangci-lint run on those packages (0 issues);
go fix ./... and golangci-lint fmt produced no diff; task docs:artifact:check.
This commit is contained in:
1 parent
0b1a852650
commit
5fdaeb72c2
14 files changed
+848
-100
No files matched your search
@@ -195,6 +195,13 @@ looking at before reporting one:
|
||||
Telegram `/restart` re-runs same TOML overlay load, then replaces only gateway
|
||||
token and allowed-user list. Every other file setting is startup-only.
|
||||
|
||||
A stored `channel-settings` change is reconciled live by `archie-messaging`:
|
||||
on a new version it re-resolves the file, layers the store over it, and restarts
|
||||
only the channel whose transport settings changed (Telegram token/allowlist,
|
||||
email relay, webhook route/secret). `chat.email.listen_addr` and
|
||||
`chat.webhook_addr` are bound listen addresses, so those changes, and enabling
|
||||
or disabling a channel, are reported outstanding and take effect on restart.
|
||||
|
||||
## Treat multi-identity as partially wired
|
||||
|
||||
`finalize` requires each identity's `name`, `bot_user`, non-empty repos, forge
|
||||
|
||||
@@ -335,17 +335,21 @@ Two properties matter to anyone changing this area:
|
||||
query, so boot records the last published settings and the reload re-applies
|
||||
them from there.
|
||||
- **Live kinds arrive on a watch.** Provider settings, model role assignments,
|
||||
repository policies and scheduling policy are `ApplyMode: live`
|
||||
(archie-core-zfb0.1): the daemon keeps a watch per kind and, on a new
|
||||
version, re-runs `boot.runtimeConfig` over the published snapshot and
|
||||
repository policies, scheduling policy, plugin settings, review settings and
|
||||
channel settings are `ApplyMode: live`: the daemon keeps a watch per kind and,
|
||||
on a new version, re-runs `boot.runtimeConfig` over the published snapshot and
|
||||
republishes through `config.Holder`. A candidate the layered document cannot
|
||||
run is refused before it is published — last-known-good keeps running and
|
||||
the refusal is reported through apply status. The kinds stay
|
||||
restart-required while a startup-built component still holds their value
|
||||
(tool, plugin, container and channel settings today); a kind joins the live
|
||||
list only with a consumer that re-reads it, and the model/provider half of
|
||||
it also rebuilds the gateway chat runtime's provider set and the chat model
|
||||
list the chat surfaces offer.
|
||||
the refusal is reported through apply status. A kind whose consumer lives in
|
||||
another process reconciles there too: `archie-messaging` re-reads the stored
|
||||
channel settings on the apply-status restamp interval and restarts only the
|
||||
channel whose transport settings changed, while the listen addresses it has
|
||||
bound and the set of channels it composed stay process bindings that take
|
||||
effect on restart. A kind joins the live list only with a consumer that
|
||||
re-reads it; one whose startup-built consumer never does stays
|
||||
restart-required, and the model/provider half of the live set also rebuilds
|
||||
the gateway chat runtime's provider set and the chat model list the chat
|
||||
surfaces offer.
|
||||
|
||||
## Completion criteria
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
"url": "/dev/architecture/configuration/",
|
||||
"title": "Configuration",
|
||||
"section": "Architecture",
|
||||
"body": "# Configuration\n\n**Status:** Approved; existing-type assignment is ongoing \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Ownership\n\nConfiguration has two separate responsibilities:\n\n1. `internal/infrastructure/configuration` loads files, environment variables,\n secret references, overlays, and compatibility representations.\n2. Each domain and plugin owns the typed runtime settings required for its\n behaviour.\n\n`internal/app` translates external input into those settings, validates\ncross-capability combinations, and supplies each component only its own\nsettings.\n\nThe following are prohibited:\n\n- passing complete application configuration into a domain;\n- making a domain depend on TOML, YAML, environment-variable names, secret\n references, or a configuration library;\n- defining a central runtime configuration type imported across the system;\n- placing plugin options in an unrelated global configuration model.\n\n## Private input document\n\n`internal/infrastructure/configuration` MAY use a complete decoded document\nwhile discovering files, decoding sections, and applying overlays. It is an\ninput-boundary DTO only.\n\nThe document:\n\n- remains private to the configuration infrastructure;\n- is composed from separately defined feature sections;\n- is immediately translated by application composition;\n- is never returned as the application's runtime model;\n- is never passed through runtime components.\n\nExternal DTOs and domain runtime settings are separate types.\n\nPlugins supply external configuration definitions through a narrow extension\ncontract. The core loader MUST NOT gain a field or recognised filename for every\nplugin.\n\nInvalid configuration input MUST NOT replace active valid settings or terminate\nunaffected capabilities. Configuration changes use the candidate validation,\npromotion, audit, and recovery requirements in\n[safe-change-and-recovery.md](/dev/architecture/safe-change-and-recovery/).\n\n## Dissolution of `internal/config`\n\n`internal/config` has no target location. It MUST be dissolved and MUST NOT\nsurvive as a shared package, configuration domain, or renamed global model.\n\nIts loading behaviour moves to `internal/infrastructure/configuration`. Existing\ntypes and methods are reassigned by behaviour. Repository execution policy,\ndispatch policy, retry policy, ecosystem defaults, tool policy, memory settings,\nprovider settings, and channel settings move to their\nrespective owners.\n\n## Repository settings\n\nThe existing `config.Repo` MUST NOT move intact into a repository domain or\nshared settings package. A managed repository is an application association:\n\n- work intake owns which repository references an identity monitors;\n- task execution owns gates, preflight, test protection, ecosystem behaviour,\n and change limits;\n- task lifecycle owns retry behaviour;\n- scheduling owns concurrency policy;\n- workspace management owns base branch and persistent-storage requirements.\n\nApplication composition correlates these settings by repository reference. No\ndomain receives a universal repository profile.\n\n## Documentation\n\nGenerated configuration reference describes external sections and their owning\ncapabilities. It MUST NOT establish a global Go configuration structure as the\npublic contract.\n\n## Value classification rule\n\nEvery value that controls behaviour MUST have an explicit owner and MUST be\nclassified as exactly one of:\n\n1. **Invariant** — changing it changes the meaning of the program, a wire\n protocol, or a domain concept. It may be a Go constant.\n2. **Runtime setting** — an operator, identity, domain, plugin, or deployment\n may legitimately choose a different value. It is typed configuration owned by\n the affected capability.\n3. **Policy value** — it expresses what is allowed, limited, routed, retried,\n protected, or gated. Its typed definition belongs to the owning domain and it\n is evaluated through the policy system.\n4. **Derived value** — it is computed from other settings and is not\n independently mutable.\n5. **Runtime state** — it changes as the application operates, such as the\n active model for a conversation. It is changed by a domain command and is not\n configuration merely because it can be changed.\n\nAn unclassified behavioural literal is a defect.\n\n`const` MUST NOT be used merely because a value currently has one default.\nConstants are reserved for actual invariants: enum discriminants, protocol\nversions, fixed compatibility tokens, and values whose alteration would change\nthe program's semantics rather than configure an installation.\n\nIn Go, a constant is a compile-time value and does not represent mutable runtime\nstorage. It generally has no address that can be observed or changed.\nArchitecturally, a constant therefore asserts that the value cannot change for\nthe lifetiLine truncated
|
||||
"body": "# Configuration\n\n**Status:** Approved; existing-type assignment is ongoing \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Ownership\n\nConfiguration has two separate responsibilities:\n\n1. `internal/infrastructure/configuration` loads files, environment variables,\n secret references, overlays, and compatibility representations.\n2. Each domain and plugin owns the typed runtime settings required for its\n behaviour.\n\n`internal/app` translates external input into those settings, validates\ncross-capability combinations, and supplies each component only its own\nsettings.\n\nThe following are prohibited:\n\n- passing complete application configuration into a domain;\n- making a domain depend on TOML, YAML, environment-variable names, secret\n references, or a configuration library;\n- defining a central runtime configuration type imported across the system;\n- placing plugin options in an unrelated global configuration model.\n\n## Private input document\n\n`internal/infrastructure/configuration` MAY use a complete decoded document\nwhile discovering files, decoding sections, and applying overlays. It is an\ninput-boundary DTO only.\n\nThe document:\n\n- remains private to the configuration infrastructure;\n- is composed from separately defined feature sections;\n- is immediately translated by application composition;\n- is never returned as the application's runtime model;\n- is never passed through runtime components.\n\nExternal DTOs and domain runtime settings are separate types.\n\nPlugins supply external configuration definitions through a narrow extension\ncontract. The core loader MUST NOT gain a field or recognised filename for every\nplugin.\n\nInvalid configuration input MUST NOT replace active valid settings or terminate\nunaffected capabilities. Configuration changes use the candidate validation,\npromotion, audit, and recovery requirements in\n[safe-change-and-recovery.md](/dev/architecture/safe-change-and-recovery/).\n\n## Dissolution of `internal/config`\n\n`internal/config` has no target location. It MUST be dissolved and MUST NOT\nsurvive as a shared package, configuration domain, or renamed global model.\n\nIts loading behaviour moves to `internal/infrastructure/configuration`. Existing\ntypes and methods are reassigned by behaviour. Repository execution policy,\ndispatch policy, retry policy, ecosystem defaults, tool policy, memory settings,\nprovider settings, and channel settings move to their\nrespective owners.\n\n## Repository settings\n\nThe existing `config.Repo` MUST NOT move intact into a repository domain or\nshared settings package. A managed repository is an application association:\n\n- work intake owns which repository references an identity monitors;\n- task execution owns gates, preflight, test protection, ecosystem behaviour,\n and change limits;\n- task lifecycle owns retry behaviour;\n- scheduling owns concurrency policy;\n- workspace management owns base branch and persistent-storage requirements.\n\nApplication composition correlates these settings by repository reference. No\ndomain receives a universal repository profile.\n\n## Documentation\n\nGenerated configuration reference describes external sections and their owning\ncapabilities. It MUST NOT establish a global Go configuration structure as the\npublic contract.\n\n## Value classification rule\n\nEvery value that controls behaviour MUST have an explicit owner and MUST be\nclassified as exactly one of:\n\n1. **Invariant** — changing it changes the meaning of the program, a wire\n protocol, or a domain concept. It may be a Go constant.\n2. **Runtime setting** — an operator, identity, domain, plugin, or deployment\n may legitimately choose a different value. It is typed configuration owned by\n the affected capability.\n3. **Policy value** — it expresses what is allowed, limited, routed, retried,\n protected, or gated. Its typed definition belongs to the owning domain and it\n is evaluated through the policy system.\n4. **Derived value** — it is computed from other settings and is not\n independently mutable.\n5. **Runtime state** — it changes as the application operates, such as the\n active model for a conversation. It is changed by a domain command and is not\n configuration merely because it can be changed.\n\nAn unclassified behavioural literal is a defect.\n\n`const` MUST NOT be used merely because a value currently has one default.\nConstants are reserved for actual invariants: enum discriminants, protocol\nversions, fixed compatibility tokens, and values whose alteration would change\nthe program's semantics rather than configure an installation.\n\nIn Go, a constant is a compile-time value and does not represent mutable runtime\nstorage. It generally has no address that can be observed or changed.\nArchitecturally, a constant therefore asserts that the value cannot change for\nthe lifetiLine truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/architecture/dashboard-agent-tools/",
|
||||
|
||||
@@ -260,15 +260,17 @@ func (b *boot) startLiveSettings(ctx context.Context) error {
|
||||
// (b.runtimeConfig, which applies the execution budgets the settings watch
|
||||
// published) and republishes through config.Holder, so the change takes
|
||||
// effect without a restart. Kinds stay out of this list while a
|
||||
// startup-built component still holds their value -- tool, container and
|
||||
// channel settings are frozen in components built at boot and remain
|
||||
// startup-built component still holds their value -- tool and container
|
||||
// settings are frozen in components built at boot and remain
|
||||
// restart-required -- and a kind joins it only with a consumer that re-reads
|
||||
// it. PluginSettingsKind is on the list because its only consumer is the
|
||||
// directory reconciliation, which reads the running config each tick; a
|
||||
// removed plugin or engine still cannot unload, so that kind reports the
|
||||
// removal rather than requiring a restart. Each kind is watched in its own
|
||||
// goroutine, exactly the shape the workflow-execution-settings watch
|
||||
// established.
|
||||
// removal rather than requiring a restart. ChannelSettingsKind is on the list
|
||||
// for this process's own re-layer; the Messaging Service, which owns the
|
||||
// channel transports, reconciles them separately and restarts only the
|
||||
// channel whose settings changed. Each kind is watched in its own goroutine,
|
||||
// exactly the shape the workflow-execution-settings watch established.
|
||||
var runtimeResourceKinds = []string{
|
||||
controlplane.ProviderSettingsKind,
|
||||
controlplane.ModelRoleAssignmentsKind,
|
||||
@@ -276,6 +278,7 @@ var runtimeResourceKinds = []string{
|
||||
controlplane.SchedulingPolicyKind,
|
||||
controlplane.PluginSettingsKind,
|
||||
controlplane.ReviewSettingsKind,
|
||||
controlplane.ChannelSettingsKind,
|
||||
}
|
||||
|
||||
// startRuntimeResourceWatches keeps a watch per live kind established for the
|
||||
|
||||
@@ -15,7 +15,7 @@ import (
|
||||
// channel. Only what an operator may see: no configuration value ever enters a
|
||||
// descriptor, so the manager cannot become a second source for a token or a
|
||||
// credential.
|
||||
func channelDescriptors(instances []channelInstance) []status.Descriptor {
|
||||
func channelDescriptors(instances []*channelInstance) []status.Descriptor {
|
||||
descriptors := make([]status.Descriptor, 0, len(instances))
|
||||
for _, instance := range instances {
|
||||
descriptors = append(descriptors, status.Descriptor{
|
||||
@@ -34,8 +34,8 @@ func channelDescriptors(instances []channelInstance) []status.Descriptor {
|
||||
// the single source for both the capability a descriptor declares and the action
|
||||
// a reload performs; deriving them separately is how a dashboard ends up offering
|
||||
// a button that reports success and changes nothing (archie-core-np7l).
|
||||
func reloadable(instance channelInstance) bool {
|
||||
gateway, ok := instance.channel.(*telegram.Gateway)
|
||||
func reloadable(instance *channelInstance) bool {
|
||||
gateway, ok := instance.current().(*telegram.Gateway)
|
||||
return ok && gateway.Reload != nil
|
||||
}
|
||||
|
||||
@@ -53,7 +53,7 @@ func (s *Service) ReloadChannel(_ context.Context, id string) error {
|
||||
if instance.name != id {
|
||||
continue
|
||||
}
|
||||
gateway, ok := instance.channel.(*telegram.Gateway)
|
||||
gateway, ok := instance.current().(*telegram.Gateway)
|
||||
if !ok || gateway.Reload == nil {
|
||||
return fmt.Errorf("messaging: channel %q does not support reload", id)
|
||||
}
|
||||
|
||||
@@ -144,7 +144,7 @@ func TestChannelDescriptorsDeclareCapabilities(t *testing.T) {
|
||||
// "telegram" correctly declares nothing.
|
||||
withReload := telegram.New("token-123", []int64{1}, slog.New(slog.DiscardHandler))
|
||||
withReload.Reload = func(*telegram.Gateway) error { return nil }
|
||||
descriptors := channelDescriptors([]channelInstance{
|
||||
descriptors := channelDescriptors([]*channelInstance{
|
||||
{name: "telegram", channel: withReload},
|
||||
{name: "email", channel: fakeChannel{}},
|
||||
{name: "webhook", channel: fakeChannel{}},
|
||||
|
||||
@@ -7,6 +7,8 @@ import (
|
||||
"net"
|
||||
"strconv"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/channels"
|
||||
"github.com/samcharles93/archie-core/internal/channels/email"
|
||||
@@ -14,6 +16,7 @@ import (
|
||||
"github.com/samcharles93/archie-core/internal/channels/telegram"
|
||||
"github.com/samcharles93/archie-core/internal/channels/webhook"
|
||||
"github.com/samcharles93/archie-core/internal/config"
|
||||
"github.com/samcharles93/archie-core/internal/domain/applystatus"
|
||||
"github.com/samcharles93/archie-core/internal/domain/health"
|
||||
"github.com/samcharles93/archie-core/internal/domain/messaging"
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
@@ -30,11 +33,44 @@ type deps struct {
|
||||
Chat messaging.ChatContract
|
||||
Health *health.Registry
|
||||
Settings *messaging.SettingsCommand
|
||||
// SettingsSource, when non-nil, enables the live channel-settings
|
||||
// reconcile: the service re-reads the stored channel-settings resource on
|
||||
// the apply-status restamp interval and restarts only the channels whose
|
||||
// settings changed. Nil leaves the service with no live path, which is the
|
||||
// honest state for a test or a process with no State Store.
|
||||
SettingsSource chatSettingsSource
|
||||
// ApplyReporter records what this process applied. Nil reports nothing.
|
||||
ApplyReporter *applystatus.Reporter
|
||||
// AppliedVersion is the channel-settings version already running when the
|
||||
// service starts, so the first reconcile tick does not re-apply it.
|
||||
AppliedVersion int64
|
||||
// ReconcileInterval overrides applystatus.RestampInterval, so a test can
|
||||
// drive the loop without waiting 30 seconds.
|
||||
ReconcileInterval time.Duration
|
||||
}
|
||||
|
||||
// channelInstance is one composed channel and the runtime state that lets the
|
||||
// service restart it alone. channel and cancel are guarded by mu; rebuild is
|
||||
// immutable. The channel swaps when the stored settings change, which is what a
|
||||
// restart is: build the replacement, stop the old run context, start the new
|
||||
// instance. A channel with no in-place reload seam is therefore still
|
||||
// reloadable.
|
||||
type channelInstance struct {
|
||||
name string
|
||||
channel channels.Channel
|
||||
rebuild func(ResolvedConfig) (channels.Channel, error)
|
||||
|
||||
mu sync.Mutex
|
||||
channel channels.Channel
|
||||
cancel context.CancelFunc
|
||||
supervised bool
|
||||
restartPending bool
|
||||
}
|
||||
|
||||
// current returns the channel instance currently serving, under the lock.
|
||||
func (c *channelInstance) current() channels.Channel {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
return c.channel
|
||||
}
|
||||
|
||||
// Service manages the lifecycle of the extracted Messaging Service and its
|
||||
@@ -55,11 +91,26 @@ type Service struct {
|
||||
log *slog.Logger
|
||||
chat messaging.ChatContract
|
||||
health *health.Registry
|
||||
channels []channelInstance
|
||||
channels []*channelInstance
|
||||
// settingsSource re-reads the stored channel settings. Nil disables the
|
||||
// reconcile loop.
|
||||
settingsSource chatSettingsSource
|
||||
// reporter records the applied channel-settings version. Nil reports
|
||||
// nothing.
|
||||
reporter *applystatus.Reporter
|
||||
// appliedVersion is the channel-settings version the running channels were
|
||||
// built from, so a reconcile tick does not re-apply its own work.
|
||||
appliedVersion atomic.Int64
|
||||
// reconcileInterval is how often the stored settings are re-read.
|
||||
reconcileInterval time.Duration
|
||||
|
||||
mu sync.Mutex
|
||||
running bool
|
||||
stop func()
|
||||
// wg tracks the per-channel supervisors, so a shutdown waits for every
|
||||
// channel to stop before Start returns. A restart can start a replacement
|
||||
// supervisor, which is why it lives here rather than in Start's frame.
|
||||
wg sync.WaitGroup
|
||||
}
|
||||
|
||||
// compose builds the Service from resolved configuration. ctx is the service
|
||||
@@ -69,57 +120,122 @@ type Service struct {
|
||||
// dropped: a front-end the operator configured must never silently not run.
|
||||
func compose(ctx context.Context, d deps) (*Service, error) {
|
||||
srv := &Service{
|
||||
cfg: d.Config,
|
||||
log: d.Log,
|
||||
chat: d.Chat,
|
||||
health: d.Health,
|
||||
cfg: d.Config,
|
||||
log: d.Log,
|
||||
chat: d.Chat,
|
||||
health: d.Health,
|
||||
settingsSource: d.SettingsSource,
|
||||
reporter: d.ApplyReporter,
|
||||
reconcileInterval: d.ReconcileInterval,
|
||||
}
|
||||
srv.appliedVersion.Store(d.AppliedVersion)
|
||||
if srv.reconcileInterval <= 0 {
|
||||
srv.reconcileInterval = applystatus.RestampInterval
|
||||
}
|
||||
|
||||
if d.Config.TelegramToken != "" {
|
||||
if len(d.Config.Telegram.AllowedUserIDs) == 0 {
|
||||
d.Log.Warn("chat.telegram has no allowed_user_ids: every sender will be rejected. " +
|
||||
"Add your Telegram user id to chat.telegram.allowed_user_ids to enable the bot.")
|
||||
}
|
||||
tg := telegram.New(d.Config.TelegramToken, d.Config.Telegram.AllowedUserIDs, d.Log)
|
||||
tg.Settings = d.Settings
|
||||
configureTelegram(ctx, tg, d.Config, d.Chat, d.Log)
|
||||
if err := srv.add("telegram", tg, telegramValidateConfigMap(d.Config.Telegram)); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
instances, err := composeChannels(ctx, d)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if d.Config.Email.ListenAddr != "" {
|
||||
em := email.New(d.Config.Email.ListenAddr, d.Config.Email.RelayAddr, d.Log)
|
||||
if err := srv.add("email", em, map[string]any{
|
||||
"listen_addr": d.Config.Email.ListenAddr,
|
||||
"relay_addr": d.Config.Email.RelayAddr,
|
||||
}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
if d.Config.WebhookAddr != "" {
|
||||
host, port := parseListenAddr(d.Config.WebhookAddr, "0.0.0.0", 8644)
|
||||
wh := webhook.New(host, port, webhookRoutes(d.Config.Webhook, d.Config.WebhookSecret), d.Log)
|
||||
if err := srv.add("webhook", wh, map[string]any{"host": host, "port": port}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
srv.channels = instances
|
||||
srv.status = status.NewManager(channelDescriptors(srv.channels))
|
||||
srv.statusWriter = d.ChannelStatus
|
||||
srv.publish = make(chan struct{}, 1)
|
||||
return srv, nil
|
||||
}
|
||||
|
||||
// add validates a channel against its own ConfigSchema contract and
|
||||
// registers it.
|
||||
func (s *Service) add(name string, ch channels.Channel, cfg map[string]any) error {
|
||||
if err := ch.ValidateConfig(cfg); err != nil {
|
||||
return fmt.Errorf("chat.%s config invalid: %w", name, err)
|
||||
// composeChannels builds the configured channel set. Each instance carries the
|
||||
// factory a restart uses to build a replacement from a new resolved
|
||||
// configuration, which is what makes a channel with no in-place reload seam
|
||||
// still restarted alone rather than through a process restart.
|
||||
func composeChannels(ctx context.Context, d deps) ([]*channelInstance, error) {
|
||||
instances := make([]*channelInstance, 0, 3)
|
||||
if d.Config.TelegramToken != "" {
|
||||
instance, err := composeTelegram(ctx, d)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
instances = append(instances, instance)
|
||||
}
|
||||
s.channels = append(s.channels, channelInstance{name: name, channel: ch})
|
||||
return nil
|
||||
if d.Config.Email.ListenAddr != "" {
|
||||
instance, err := composeEmail(d)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
instances = append(instances, instance)
|
||||
}
|
||||
if d.Config.WebhookAddr != "" {
|
||||
instance, err := composeWebhook(d)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
instances = append(instances, instance)
|
||||
}
|
||||
return instances, nil
|
||||
}
|
||||
|
||||
func composeTelegram(ctx context.Context, d deps) (*channelInstance, error) {
|
||||
if len(d.Config.Telegram.AllowedUserIDs) == 0 {
|
||||
d.Log.Warn("chat.telegram has no allowed_user_ids: every sender will be rejected. " +
|
||||
"Add your Telegram user id to chat.telegram.allowed_user_ids to enable the bot.")
|
||||
}
|
||||
build := func(cfg ResolvedConfig) (channels.Channel, error) {
|
||||
if cfg.TelegramToken == "" {
|
||||
return nil, nil
|
||||
}
|
||||
tg := telegram.New(cfg.TelegramToken, cfg.Telegram.AllowedUserIDs, d.Log)
|
||||
tg.Settings = d.Settings
|
||||
configureTelegram(ctx, tg, cfg, d.Chat, d.Log)
|
||||
if err := tg.ValidateConfig(telegramValidateConfigMap(cfg.Telegram)); err != nil {
|
||||
return nil, fmt.Errorf("chat.telegram config invalid: %w", err)
|
||||
}
|
||||
return tg, nil
|
||||
}
|
||||
ch, err := build(d.Config)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &channelInstance{name: "telegram", channel: ch, rebuild: build}, nil
|
||||
}
|
||||
|
||||
func composeEmail(d deps) (*channelInstance, error) {
|
||||
build := func(cfg ResolvedConfig) (channels.Channel, error) {
|
||||
if cfg.Email.ListenAddr == "" {
|
||||
return nil, nil
|
||||
}
|
||||
em := email.New(cfg.Email.ListenAddr, cfg.Email.RelayAddr, d.Log)
|
||||
if err := em.ValidateConfig(map[string]any{
|
||||
"listen_addr": cfg.Email.ListenAddr,
|
||||
"relay_addr": cfg.Email.RelayAddr,
|
||||
}); err != nil {
|
||||
return nil, fmt.Errorf("chat.email config invalid: %w", err)
|
||||
}
|
||||
return em, nil
|
||||
}
|
||||
ch, err := build(d.Config)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &channelInstance{name: "email", channel: ch, rebuild: build}, nil
|
||||
}
|
||||
|
||||
func composeWebhook(d deps) (*channelInstance, error) {
|
||||
build := func(cfg ResolvedConfig) (channels.Channel, error) {
|
||||
if cfg.WebhookAddr == "" {
|
||||
return nil, nil
|
||||
}
|
||||
host, port := parseListenAddr(cfg.WebhookAddr, "0.0.0.0", 8644)
|
||||
wh := webhook.New(host, port, webhookRoutes(cfg.Webhook, cfg.WebhookSecret), d.Log)
|
||||
if err := wh.ValidateConfig(map[string]any{"host": host, "port": port}); err != nil {
|
||||
return nil, fmt.Errorf("chat.webhook config invalid: %w", err)
|
||||
}
|
||||
return wh, nil
|
||||
}
|
||||
ch, err := build(d.Config)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &channelInstance{name: "webhook", channel: ch, rebuild: build}, nil
|
||||
}
|
||||
|
||||
// telegramValidateConfigMap builds the map Gateway.ValidateConfig expects
|
||||
@@ -180,24 +296,17 @@ func (s *Service) Start(ctx context.Context) error {
|
||||
// opened during startup shows configured channels rather than none.
|
||||
s.signalPublish()
|
||||
}
|
||||
if s.settingsSource != nil {
|
||||
go s.reconcileLoop(ctx)
|
||||
}
|
||||
|
||||
var wg sync.WaitGroup
|
||||
for _, ch := range s.channels {
|
||||
c := ch
|
||||
wg.Go(func() {
|
||||
s.log.Info("starting channel", "name", c.name)
|
||||
if err := c.channel.Start(ctx, s.chat, s.lifecycleFor(c.name)); err != nil && ctx.Err() == nil {
|
||||
// The channel could not run: say so on the operator's surface, not
|
||||
// only in the log, which is the difference between a dashboard that
|
||||
// shows a dead channel and one that shows nothing at all.
|
||||
s.status.MarkFailed(c.name, err.Error())
|
||||
s.log.Error("channel stopped with error", "name", c.name, "err", err)
|
||||
return
|
||||
}
|
||||
// A channel that returns without an error has stopped, either because
|
||||
// the service was asked to or because its own loop ended.
|
||||
s.status.MarkStopped(c.name, "")
|
||||
})
|
||||
c.mu.Lock()
|
||||
c.supervised = true
|
||||
c.mu.Unlock()
|
||||
s.wg.Add(1)
|
||||
go s.runChannel(ctx, c)
|
||||
}
|
||||
|
||||
// Every configured channel has been handed its lifetime and the service
|
||||
@@ -214,18 +323,10 @@ func (s *Service) Start(ctx context.Context) error {
|
||||
|
||||
<-ctx.Done()
|
||||
|
||||
for _, ch := range s.channels {
|
||||
shutdownCtx, shutdownCancel := context.WithTimeout(context.WithoutCancel(ctx), s.cfg.Options.ShutdownTimeout)
|
||||
if err := ch.channel.Stop(shutdownCtx); err != nil {
|
||||
s.log.Warn("channel stop failed", "name", ch.name, "err", err)
|
||||
}
|
||||
shutdownCancel()
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
s.wg.Wait()
|
||||
|
||||
if s.statusWriter != nil {
|
||||
// The final report carries the stopped states the loop above recorded.
|
||||
// The final report carries the stopped states each supervisor recorded.
|
||||
// Its own context, because ctx is already cancelled.
|
||||
finalCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), s.cfg.Options.ShutdownTimeout)
|
||||
defer cancel()
|
||||
|
||||
@@ -0,0 +1,216 @@
|
||||
package archiemessaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"slices"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/channels/telegram"
|
||||
"github.com/samcharles93/archie-core/internal/config"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanerpc"
|
||||
"github.com/samcharles93/archie-core/internal/secret"
|
||||
)
|
||||
|
||||
// chatSettingsSource reads the stored channel-settings resource and layers it
|
||||
// over the file document. *controlplanerpc.Client satisfies it.
|
||||
type chatSettingsSource interface {
|
||||
RuntimeChatConfig(ctx context.Context, base config.ChatConfig) (config.ChatConfig, int64, error)
|
||||
}
|
||||
|
||||
// reconcileLoop re-reads the stored channel settings on the apply-status
|
||||
// restamp interval until ctx ends. A poll rather than a stream: the restamp
|
||||
// interval is the cadence every process already reports at, the messaging
|
||||
// process has no other control-plane watch to share a reconnect ladder with,
|
||||
// and a stored document that cannot be read must leave the running channels
|
||||
// alone (docs/prds/plugin-settings-live.md).
|
||||
func (s *Service) reconcileLoop(ctx context.Context) {
|
||||
ticker := time.NewTicker(s.reconcileInterval)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
if err := s.reconcileOnce(ctx); err != nil {
|
||||
s.log.Warn("channel settings reconcile failed", "err", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// reconcileOnce re-reads the stored channel settings and, when the version
|
||||
// moved, applies the change to the running channels and records the outcome in
|
||||
// apply status.
|
||||
//
|
||||
// The file document is re-resolved first, exactly as the /restart path does, so
|
||||
// the layering is the same projection boot used: the store overrides the file,
|
||||
// and a field the store omits keeps the file's value.
|
||||
func (s *Service) reconcileOnce(ctx context.Context) error {
|
||||
base, err := Resolve(s.currentConfig().Options, s.log)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
layered, version, err := s.settingsSource.RuntimeChatConfig(ctx, config.ChatConfig{
|
||||
Telegram: base.Telegram, Email: base.Email, Webhook: base.Webhook,
|
||||
WebhookAddr: base.WebhookAddr, ShowToolCalls: base.ShowToolCalls,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if version == 0 || version <= s.appliedVersion.Load() {
|
||||
return nil
|
||||
}
|
||||
next, err := resolveChatSecrets(base, layered)
|
||||
if err != nil {
|
||||
// A credential this process cannot resolve is a document rejection: the
|
||||
// running channels keep serving and the version stays reported as not
|
||||
// applied, so the operator sees the refusal rather than a false success.
|
||||
s.report(ctx, version, err)
|
||||
return err
|
||||
}
|
||||
applied, outstanding := s.applyChannelSettings(next)
|
||||
if applied {
|
||||
s.appliedVersion.Store(version)
|
||||
}
|
||||
s.report(ctx, version, outstanding)
|
||||
return nil
|
||||
}
|
||||
|
||||
// resolveChatSecrets carries the layered chat settings onto the resolved
|
||||
// projection, resolving the store's secret references. A channel whose token
|
||||
// cannot be resolved stays off rather than composing with an empty credential
|
||||
// (the startup rule, config.go resolveTelegramToken).
|
||||
func resolveChatSecrets(base ResolvedConfig, layered config.ChatConfig) (ResolvedConfig, error) {
|
||||
next := base
|
||||
next.Telegram, next.Email, next.Webhook = layered.Telegram, layered.Email, layered.Webhook
|
||||
next.WebhookAddr, next.ShowToolCalls = layered.WebhookAddr, layered.ShowToolCalls
|
||||
|
||||
secrets := secret.NewRegistry()
|
||||
token, err := resolveTelegramToken(layered.Telegram, secrets)
|
||||
if err != nil {
|
||||
return next, fmt.Errorf("resolve database telegram token: %w", err)
|
||||
}
|
||||
next.TelegramToken = token
|
||||
webhookSecret, err := resolveWebhookSecret(layered.Webhook, secrets)
|
||||
if err != nil {
|
||||
return next, fmt.Errorf("resolve database webhook secret: %w", err)
|
||||
}
|
||||
next.WebhookSecret = webhookSecret
|
||||
return next, nil
|
||||
}
|
||||
|
||||
// applyChannelSettings applies a freshly resolved chat config to the running
|
||||
// channels. A channel restarts only when its own transport settings changed;
|
||||
// every other channel keeps serving. It reports whether every requested restart
|
||||
// was carried out, so a rebuild that failed is retried rather than recorded as
|
||||
// applied.
|
||||
//
|
||||
// Machine-level settings do not restart a channel. The chat session defaults
|
||||
// (workspace, filesystem access, max steps, rate limit) are not part of the
|
||||
// messaging projection at all, and show_tool_calls is handed to the running
|
||||
// Telegram gateway in place, so both apply to turns that start after the change
|
||||
// rather than interrupting one in flight.
|
||||
//
|
||||
// The composed set and the listen addresses are fixed for the life of the
|
||||
// process. Enabling or disabling a channel and rebinding a listen address are
|
||||
// process bindings; the change is refused and reported for a restart instead of
|
||||
// applied (docs/prds/runtime-control-plane.md, "Some changes require a
|
||||
// restart").
|
||||
func (s *Service) applyChannelSettings(next ResolvedConfig) (bool, error) {
|
||||
current := s.currentConfig()
|
||||
effective, outstanding := pinProcessBindings(current, next)
|
||||
|
||||
if next.ShowToolCalls != current.ShowToolCalls {
|
||||
s.applyShowToolCalls(next.ShowToolCalls)
|
||||
}
|
||||
|
||||
applied := true
|
||||
for _, id := range channelChanges(current, effective) {
|
||||
if err := s.restartChannel(id, effective); err != nil {
|
||||
outstanding = append(outstanding, err.Error())
|
||||
applied = false
|
||||
}
|
||||
}
|
||||
// A failed restart leaves the running configuration in place, so the next
|
||||
// tick sees the same change and retries it. A change that was carried out
|
||||
// (including a binding change that was refused and reported) is recorded, so
|
||||
// it is not applied again on every tick.
|
||||
if applied {
|
||||
s.setConfig(effective)
|
||||
}
|
||||
if len(outstanding) == 0 {
|
||||
return applied, nil
|
||||
}
|
||||
return applied, errors.New(strings.Join(outstanding, "; "))
|
||||
}
|
||||
|
||||
// applyShowToolCalls updates the running Telegram gateways in place. Nothing
|
||||
// else consumes the setting, and it is read per reply, so it needs no restart.
|
||||
func (s *Service) applyShowToolCalls(show bool) {
|
||||
for _, c := range s.channels {
|
||||
if g, ok := c.current().(*telegram.Gateway); ok {
|
||||
g.SetShowToolCalls(show)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// pinProcessBindings holds the values a running process cannot change in place:
|
||||
// the listen addresses it already bound, and the set of channels it composed.
|
||||
// The returned configuration keeps the running value for each, and the returned
|
||||
// problems name what the operator changed and when it takes effect.
|
||||
func pinProcessBindings(current, next ResolvedConfig) (ResolvedConfig, []string) {
|
||||
effective := next
|
||||
var problems []string
|
||||
|
||||
if (current.TelegramToken != "") != (next.TelegramToken != "") {
|
||||
problems = append(problems, "chat.telegram enabled/disabled; takes effect on restart")
|
||||
effective.Telegram, effective.TelegramToken = current.Telegram, current.TelegramToken
|
||||
}
|
||||
if (current.Email.ListenAddr != "") != (next.Email.ListenAddr != "") {
|
||||
problems = append(problems, "chat.email enabled/disabled; takes effect on restart")
|
||||
effective.Email = current.Email
|
||||
}
|
||||
if (current.WebhookAddr != "") != (next.WebhookAddr != "") {
|
||||
problems = append(problems, "chat.webhook enabled/disabled; takes effect on restart")
|
||||
effective.Webhook, effective.WebhookAddr, effective.WebhookSecret = current.Webhook, current.WebhookAddr, current.WebhookSecret
|
||||
}
|
||||
|
||||
if current.Email.ListenAddr != "" && next.Email.ListenAddr != current.Email.ListenAddr {
|
||||
problems = append(problems, "chat.email.listen_addr changed; takes effect on restart")
|
||||
effective.Email.ListenAddr = current.Email.ListenAddr
|
||||
}
|
||||
if current.WebhookAddr != "" && next.WebhookAddr != current.WebhookAddr {
|
||||
problems = append(problems, "chat.webhook_addr changed; takes effect on restart")
|
||||
effective.WebhookAddr = current.WebhookAddr
|
||||
}
|
||||
return effective, problems
|
||||
}
|
||||
|
||||
// channelChanges returns the composed channels whose transport settings differ
|
||||
// between the running and the effective configuration. A channel absent from
|
||||
// current (not composed) never appears, and an unchanged channel is not
|
||||
// restarted.
|
||||
func channelChanges(current, next ResolvedConfig) []string {
|
||||
var changed []string
|
||||
if current.TelegramToken != "" &&
|
||||
(current.TelegramToken != next.TelegramToken || !slices.Equal(current.Telegram.AllowedUserIDs, next.Telegram.AllowedUserIDs)) {
|
||||
changed = append(changed, "telegram")
|
||||
}
|
||||
if current.Email.ListenAddr != "" && current.Email.RelayAddr != next.Email.RelayAddr {
|
||||
changed = append(changed, "email")
|
||||
}
|
||||
if current.WebhookAddr != "" && (current.Webhook != next.Webhook || current.WebhookSecret != next.WebhookSecret) {
|
||||
changed = append(changed, "webhook")
|
||||
}
|
||||
return changed
|
||||
}
|
||||
|
||||
func (s *Service) report(ctx context.Context, version int64, applyErr error) {
|
||||
if s.reporter == nil {
|
||||
return
|
||||
}
|
||||
s.reporter.Report(ctx, controlplanerpc.ChannelSettingsKind, version, applyErr)
|
||||
}
|
||||
@@ -0,0 +1,253 @@
|
||||
package archiemessaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/channels/status"
|
||||
"github.com/samcharles93/archie-core/internal/config"
|
||||
"github.com/samcharles93/archie-core/internal/domain/applystatus"
|
||||
"github.com/samcharles93/archie-core/internal/domain/messaging"
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
)
|
||||
|
||||
// channelByID returns the live instance the service currently serves for id.
|
||||
func channelByID(t *testing.T, srv *Service, id string) *channelInstance {
|
||||
t.Helper()
|
||||
for _, instance := range srv.channels {
|
||||
if instance.name == id {
|
||||
return instance
|
||||
}
|
||||
}
|
||||
t.Fatalf("no channel %q composed", id)
|
||||
return nil
|
||||
}
|
||||
|
||||
// channelAddrs reserves two loopback ports, one per channel.
|
||||
func channelAddrs(t *testing.T) (emailAddr, webhookAddr string) {
|
||||
t.Helper()
|
||||
return fmt.Sprintf("127.0.0.1:%d", freePort(t)), fmt.Sprintf("127.0.0.1:%d", freePort(t))
|
||||
}
|
||||
|
||||
// writeChannelConfig writes a file document with email and webhook enabled. A
|
||||
// file rather than a hand-built ResolvedConfig, because the reconcile path
|
||||
// re-resolves the file exactly as production does; a hand-built config would
|
||||
// prove nothing about that half.
|
||||
func writeChannelConfig(t *testing.T, emailAddr, webhookAddr, route string) string {
|
||||
t.Helper()
|
||||
path := filepath.Join(t.TempDir(), "config.toml")
|
||||
content := fmt.Sprintf("bot_user = \"testbot\"\n\n[chat]\nwebhook_addr = %q\n\n[chat.email]\nlisten_addr = %q\n\n[chat.webhook]\npath = %q\n",
|
||||
webhookAddr, emailAddr, route)
|
||||
if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return path
|
||||
}
|
||||
|
||||
func resolveChannels(t *testing.T, emailAddr, webhookAddr, route string) ResolvedConfig {
|
||||
t.Helper()
|
||||
cfg, err := Resolve(
|
||||
Options{Config: writeChannelConfig(t, emailAddr, webhookAddr, route), ShutdownTimeout: 2 * time.Second},
|
||||
slog.New(slog.DiscardHandler),
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("Resolve: %v", err)
|
||||
}
|
||||
return cfg
|
||||
}
|
||||
|
||||
// startComposed starts a composed service and waits for its channels to run.
|
||||
func startComposed(t *testing.T, d deps) *Service {
|
||||
t.Helper()
|
||||
srv, err := compose(t.Context(), d)
|
||||
if err != nil {
|
||||
t.Fatalf("compose: %v", err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(t.Context())
|
||||
started := make(chan error, 1)
|
||||
go func() { started <- srv.Start(ctx) }()
|
||||
t.Cleanup(func() {
|
||||
srv.Stop()
|
||||
cancel()
|
||||
select {
|
||||
case err := <-started:
|
||||
if err != nil {
|
||||
t.Errorf("srv.Start returned %v, want a clean stop", err)
|
||||
}
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Error("srv.Start did not return after Stop")
|
||||
}
|
||||
})
|
||||
waitForState(t, srv, "email", status.StateRunning)
|
||||
waitForState(t, srv, "webhook", status.StateRunning)
|
||||
return srv
|
||||
}
|
||||
|
||||
// startChannels composes and starts email and webhook. Those two are the
|
||||
// channels a test can run without an external service, and a restart of one is
|
||||
// observable as the channel pointer changing.
|
||||
func startChannels(t *testing.T) *Service {
|
||||
t.Helper()
|
||||
emailAddr, webhookAddr := channelAddrs(t)
|
||||
return startComposed(t, deps{
|
||||
Config: resolveChannels(t, emailAddr, webhookAddr, "/a"),
|
||||
Log: slog.New(slog.DiscardHandler),
|
||||
Chat: &recordingChatContract{routes: make(chan messaging.Inbound, 1), reply: "ok"},
|
||||
})
|
||||
}
|
||||
|
||||
// TestApplyChannelSettingsRestartsOnlyTheChangedChannel is the bead's core
|
||||
// property: a change to one channel restarts that channel and leaves the others
|
||||
// serving. Before the per-channel supervisor this could only be a process
|
||||
// restart, which restarted every channel (archie-core-zfb0.4).
|
||||
func TestApplyChannelSettingsRestartsOnlyTheChangedChannel(t *testing.T) {
|
||||
srv := startChannels(t)
|
||||
webhookBefore := channelByID(t, srv, "webhook").current()
|
||||
emailBefore := channelByID(t, srv, "email").current()
|
||||
|
||||
next := srv.currentConfig()
|
||||
next.Webhook = config.WebhookRoute{Path: "/b"}
|
||||
if _, err := srv.applyChannelSettings(next); err != nil {
|
||||
t.Fatalf("applyChannelSettings: %v", err)
|
||||
}
|
||||
|
||||
if got := channelByID(t, srv, "webhook").current(); got == webhookBefore {
|
||||
t.Error("webhook did not restart after its route changed")
|
||||
}
|
||||
if got := channelByID(t, srv, "email").current(); got != emailBefore {
|
||||
t.Error("email restarted when only the webhook route changed")
|
||||
}
|
||||
}
|
||||
|
||||
// TestApplyChannelSettingsRefusesAListenAddressChange is the path where the
|
||||
// change must NOT apply: a bound listen address is a process binding, and
|
||||
// rebinding it in place is exactly the restart the epic keeps. The running
|
||||
// channel is left alone and the change is reported instead.
|
||||
func TestApplyChannelSettingsRefusesAListenAddressChange(t *testing.T) {
|
||||
srv := startChannels(t)
|
||||
before := channelByID(t, srv, "webhook").current()
|
||||
|
||||
next := srv.currentConfig()
|
||||
next.WebhookAddr = fmt.Sprintf("127.0.0.1:%d", freePort(t))
|
||||
_, err := srv.applyChannelSettings(next)
|
||||
if err == nil {
|
||||
t.Fatal("applyChannelSettings error = nil, want the listen-address change reported")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "webhook_addr") {
|
||||
t.Errorf("error = %v, want it to name chat.webhook_addr", err)
|
||||
}
|
||||
if got := channelByID(t, srv, "webhook").current(); got != before {
|
||||
t.Error("webhook restarted for a listen-address change; a bound address must not move in place")
|
||||
}
|
||||
}
|
||||
|
||||
// TestApplyChannelSettingsRestartsNothingWhenNothingChanged: the other half of
|
||||
// "only the channels whose settings changed" -- a tick that carries no change
|
||||
// must not restart a healthy channel.
|
||||
func TestApplyChannelSettingsRestartsNothingWhenNothingChanged(t *testing.T) {
|
||||
srv := startChannels(t)
|
||||
webhookBefore := channelByID(t, srv, "webhook").current()
|
||||
emailBefore := channelByID(t, srv, "email").current()
|
||||
|
||||
if _, err := srv.applyChannelSettings(srv.currentConfig()); err != nil {
|
||||
t.Fatalf("applyChannelSettings: %v", err)
|
||||
}
|
||||
if channelByID(t, srv, "webhook").current() != webhookBefore {
|
||||
t.Error("webhook restarted with no settings change")
|
||||
}
|
||||
if channelByID(t, srv, "email").current() != emailBefore {
|
||||
t.Error("email restarted with no settings change")
|
||||
}
|
||||
}
|
||||
|
||||
// fakeChannelSettings is the stored channel-settings resource a test owns. It
|
||||
// returns the document a real store would: the file's values with the stored
|
||||
// route layered over them, so the reconcile's layering is exercised rather than
|
||||
// bypassed.
|
||||
type fakeChannelSettings struct {
|
||||
mu sync.Mutex
|
||||
settings config.ChatConfig
|
||||
version int64
|
||||
}
|
||||
|
||||
func (f *fakeChannelSettings) RuntimeChatConfig(_ context.Context, base config.ChatConfig) (config.ChatConfig, int64, error) {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
out := base
|
||||
out.Webhook = f.settings.Webhook
|
||||
return out, f.version, nil
|
||||
}
|
||||
|
||||
func (f *fakeChannelSettings) set(version int64, route config.WebhookRoute) {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
f.version, f.settings.Webhook = version, route
|
||||
}
|
||||
|
||||
// recordingApplyStore records the apply-status writes the service makes.
|
||||
type recordingApplyStore struct {
|
||||
mu sync.Mutex
|
||||
writes []storecontract.ApplyStatus
|
||||
}
|
||||
|
||||
func (s *recordingApplyStore) PutApplyStatus(_ context.Context, st storecontract.ApplyStatus) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
s.writes = append(s.writes, st)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *recordingApplyStore) ListApplyStatus(context.Context) ([]storecontract.ApplyStatus, error) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (s *recordingApplyStore) appliedVersion() int64 {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
var version int64
|
||||
for _, st := range s.writes {
|
||||
version = max(version, st.AppliedVersion)
|
||||
}
|
||||
return version
|
||||
}
|
||||
|
||||
// TestReconcileAppliesANewStoredVersion drives the whole live path: the service
|
||||
// reads the stored channel settings on its interval, sees a newer version, and
|
||||
// restarts the channel it changed. The applied version reaches apply status, so
|
||||
// the dashboard stops showing the change as pending.
|
||||
func TestReconcileAppliesANewStoredVersion(t *testing.T) {
|
||||
emailAddr, webhookAddr := channelAddrs(t)
|
||||
source := &fakeChannelSettings{}
|
||||
source.set(1, config.WebhookRoute{Path: "/a"})
|
||||
store := &recordingApplyStore{}
|
||||
|
||||
srv := startComposed(t, deps{
|
||||
Config: resolveChannels(t, emailAddr, webhookAddr, "/a"),
|
||||
Log: slog.New(slog.DiscardHandler),
|
||||
Chat: &recordingChatContract{routes: make(chan messaging.Inbound, 1), reply: "ok"},
|
||||
SettingsSource: source,
|
||||
ApplyReporter: applystatus.New(applystatus.Messaging, store, slog.New(slog.DiscardHandler)),
|
||||
AppliedVersion: 1,
|
||||
ReconcileInterval: 5 * time.Millisecond,
|
||||
})
|
||||
|
||||
before := channelByID(t, srv, "webhook").current()
|
||||
source.set(2, config.WebhookRoute{Path: "/b"})
|
||||
|
||||
deadline := time.Now().Add(3 * time.Second)
|
||||
for time.Now().Before(deadline) {
|
||||
if channelByID(t, srv, "webhook").current() != before && store.appliedVersion() >= 2 {
|
||||
return
|
||||
}
|
||||
time.Sleep(5 * time.Millisecond)
|
||||
}
|
||||
t.Fatalf("after the stored version moved to 2: channel replaced=%v, applied version=%d; want both",
|
||||
channelByID(t, srv, "webhook").current() != before, store.appliedVersion())
|
||||
}
|
||||
@@ -36,6 +36,13 @@ func Run(ctx context.Context, o Options) error {
|
||||
// channelStatusStore is the same client, held where compose can reach it: the
|
||||
// channel report is published to the store this process already dials.
|
||||
var channelStatusStore storecontract.ChannelStatusStore
|
||||
// settingsSource and applyReporter are the live channel-settings path: the
|
||||
// control-plane client to re-read the resource, and the record of what this
|
||||
// process applied. Both stay nil without a State Store, which leaves the
|
||||
// service with no live path.
|
||||
var settingsSource chatSettingsSource
|
||||
var applyReporter *applystatus.Reporter
|
||||
var appliedVersion int64
|
||||
if cfg.Options.StateStore.Target != "" {
|
||||
stateStore, closeClient, dialErr := staterpc.Dial(cfg.Options.StateStore.Target, cfg.Options.StateStore.Token)
|
||||
if dialErr != nil {
|
||||
@@ -43,7 +50,8 @@ func Run(ctx context.Context, o Options) error {
|
||||
}
|
||||
closeStateStore = closeClient
|
||||
defer closeStateStore()
|
||||
chatSettings, channelVersion, settingsErr := controlplanerpc.NewRPCClient(stateStore.ControlPlane()).RuntimeChatConfig(ctx, config.ChatConfig{
|
||||
controlPlaneClient := controlplanerpc.NewRPCClient(stateStore.ControlPlane())
|
||||
chatSettings, channelVersion, settingsErr := controlPlaneClient.RuntimeChatConfig(ctx, config.ChatConfig{
|
||||
Telegram: cfg.Telegram, Email: cfg.Email, Webhook: cfg.Webhook, WebhookAddr: cfg.WebhookAddr, ShowToolCalls: cfg.ShowToolCalls,
|
||||
})
|
||||
if settingsErr != nil {
|
||||
@@ -64,15 +72,19 @@ func Run(ctx context.Context, o Options) error {
|
||||
}
|
||||
settings = messaging.NewSettingsCommand(messagingControlPlane{client: stateStore.ControlPlane()}).WithIdentities(stateStore)
|
||||
channelStatusStore = stateStore
|
||||
settingsSource, applyReporter, appliedVersion = controlPlaneClient, reporter, channelVersion
|
||||
}
|
||||
|
||||
srv, err := compose(ctx, deps{
|
||||
ChannelStatus: channelStatusStore,
|
||||
Config: cfg,
|
||||
Log: log,
|
||||
Chat: chat,
|
||||
Health: health,
|
||||
Settings: settings,
|
||||
ChannelStatus: channelStatusStore,
|
||||
Config: cfg,
|
||||
Log: log,
|
||||
Chat: chat,
|
||||
Health: health,
|
||||
Settings: settings,
|
||||
SettingsSource: settingsSource,
|
||||
ApplyReporter: applyReporter,
|
||||
AppliedVersion: appliedVersion,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
package archiemessaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/channels"
|
||||
)
|
||||
|
||||
// runChannel supervises one channel for the life of the service. It starts the
|
||||
// channel and, when the channel stops because a restart was requested, starts
|
||||
// the replacement. A channel that returns on its own is recorded and not
|
||||
// restarted, so a channel whose configuration cannot run does not spin.
|
||||
//
|
||||
// The supervisor is per channel rather than one loop over every channel
|
||||
// because a channel-settings change must restart exactly the channel whose
|
||||
// settings changed: cancelling one run context stops one channel and leaves the
|
||||
// rest serving (archie-core-zfb0.4).
|
||||
func (s *Service) runChannel(ctx context.Context, c *channelInstance) {
|
||||
defer s.wg.Done()
|
||||
for {
|
||||
c.mu.Lock()
|
||||
ch := c.channel
|
||||
runCtx, cancel := context.WithCancel(ctx)
|
||||
c.cancel = cancel
|
||||
c.mu.Unlock()
|
||||
|
||||
s.log.Info("starting channel", "name", c.name)
|
||||
err := ch.Start(runCtx, s.chat, s.lifecycleFor(c.name))
|
||||
|
||||
// Decide the next state under the same lock a restart takes, so a
|
||||
// restart racing this exit is either seen here (restartPending) or
|
||||
// observed as a channel that is no longer supervised.
|
||||
c.mu.Lock()
|
||||
c.cancel = nil
|
||||
shutdown := ctx.Err() != nil
|
||||
restart := c.restartPending
|
||||
if !shutdown && restart {
|
||||
c.restartPending = false
|
||||
}
|
||||
if shutdown || !restart {
|
||||
c.supervised = false
|
||||
}
|
||||
c.mu.Unlock()
|
||||
cancel()
|
||||
|
||||
// Stop is safe to call after the run context was cancelled: each
|
||||
// channel's Start returns having closed what it opened, and Stop is the
|
||||
// idempotent tail for the paths that leave a listener behind.
|
||||
s.stopChannel(ctx, c.name, ch)
|
||||
|
||||
if shutdown {
|
||||
if err != nil && !errors.Is(err, context.Canceled) {
|
||||
s.status.MarkFailed(c.name, err.Error())
|
||||
} else {
|
||||
s.status.MarkStopped(c.name, "")
|
||||
}
|
||||
return
|
||||
}
|
||||
if restart {
|
||||
// The replacement is already in c.channel (see restartChannel); the
|
||||
// next pass starts it.
|
||||
continue
|
||||
}
|
||||
// The channel stopped on its own.
|
||||
if err != nil {
|
||||
s.status.MarkFailed(c.name, err.Error())
|
||||
s.log.Error("channel stopped with error", "name", c.name, "err", err)
|
||||
} else {
|
||||
s.status.MarkStopped(c.name, "")
|
||||
}
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
// restartChannel stops the running channel id and starts a replacement built
|
||||
// from next. The replacement is built before the running channel is touched, so
|
||||
// a configuration this build cannot run leaves the running channel serving
|
||||
// (the validate-before-switch rule in docs/prds/runtime-control-plane.md).
|
||||
//
|
||||
// A channel that is not currently supervised is not restarted: it already
|
||||
// stopped on its own, and resurrecting it here would race that exit. The caller
|
||||
// reports it.
|
||||
func (s *Service) restartChannel(id string, next ResolvedConfig) error {
|
||||
c := s.instanceByID(id)
|
||||
if c == nil {
|
||||
return fmt.Errorf("no channel %q", id)
|
||||
}
|
||||
if c.rebuild == nil {
|
||||
return fmt.Errorf("channel %q cannot be rebuilt", id)
|
||||
}
|
||||
replacement, err := c.rebuild(next)
|
||||
if err != nil {
|
||||
return fmt.Errorf("rebuild %q: %w", id, err)
|
||||
}
|
||||
if replacement == nil {
|
||||
return fmt.Errorf("channel %q has no configuration", id)
|
||||
}
|
||||
c.mu.Lock()
|
||||
if !c.supervised {
|
||||
c.mu.Unlock()
|
||||
return fmt.Errorf("channel %q is not running", id)
|
||||
}
|
||||
c.channel = replacement
|
||||
c.restartPending = true
|
||||
cancel := c.cancel
|
||||
c.mu.Unlock()
|
||||
if cancel != nil {
|
||||
cancel()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// stopChannel closes one channel's listener with its own bounded context, so a
|
||||
// stop that hangs cannot hold the shutdown past the configured timeout. The
|
||||
// parent context is stripped of its cancellation because this runs on the
|
||||
// shutdown path, where the parent is already cancelled.
|
||||
func (s *Service) stopChannel(ctx context.Context, name string, ch channels.Channel) {
|
||||
timeout := s.currentConfig().Options.ShutdownTimeout
|
||||
if timeout <= 0 {
|
||||
timeout = defaultShutdownTimeout
|
||||
}
|
||||
stopCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), timeout)
|
||||
defer cancel()
|
||||
if err := ch.Stop(stopCtx); err != nil {
|
||||
s.log.Warn("channel stop failed", "name", name, "err", err)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) instanceByID(id string) *channelInstance {
|
||||
for _, c := range s.channels {
|
||||
if c.name == id {
|
||||
return c
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Service) currentConfig() ResolvedConfig {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
return s.cfg
|
||||
}
|
||||
|
||||
func (s *Service) setConfig(cfg ResolvedConfig) {
|
||||
s.mu.Lock()
|
||||
s.cfg = cfg
|
||||
s.mu.Unlock()
|
||||
}
|
||||
@@ -41,7 +41,7 @@ type pluginSettings struct {
|
||||
func operationalDefinitions() []Definition {
|
||||
return []Definition{
|
||||
{Kind: RepositoryPoliciesKind, Title: "Repository policies", ApplyMode: "live", Document: []config.Repo{}, Seed: func(cfg config.Config) any { return cfg.Repos }, Validate: validateRepositories},
|
||||
{Kind: ChannelSettingsKind, Title: "Channel settings", ApplyMode: "restart-required", Document: channelSettings{}, Seed: seedChannels, Validate: validateChannels, Normalize: normalizeChannels},
|
||||
{Kind: ChannelSettingsKind, Title: "Channel settings", ApplyMode: "live", Document: channelSettings{}, Seed: seedChannels, Validate: validateChannels, Normalize: normalizeChannels},
|
||||
{Kind: SchedulingPolicyKind, Title: "Scheduling policy", ApplyMode: "live", Document: schedulingPolicy{}, Seed: func(cfg config.Config) any {
|
||||
return schedulingPolicy{PollInterval: cfg.PollInterval.Std().String(), MaxRetries: cfg.MaxRetries, Label: &cfg.Label, Dispatch: cfg.Dispatch}
|
||||
}, Validate: validateScheduling},
|
||||
|
||||
@@ -553,13 +553,13 @@ func TestRuntimeResourceKindsApplyLive(t *testing.T) {
|
||||
modes[definition.Kind] = definition.ApplyMode
|
||||
}
|
||||
for _, kind := range []string{
|
||||
ProviderSettingsKind, ModelRoleAssignmentsKind, RepositoryPoliciesKind, SchedulingPolicyKind, AgentProfileKind, CredentialBindingsKind, PluginSettingsKind, ReviewSettingsKind,
|
||||
ProviderSettingsKind, ModelRoleAssignmentsKind, RepositoryPoliciesKind, SchedulingPolicyKind, AgentProfileKind, CredentialBindingsKind, PluginSettingsKind, ReviewSettingsKind, ChannelSettingsKind,
|
||||
} {
|
||||
if modes[kind] != "live" {
|
||||
t.Errorf("%s applies %q, want live: the daemon re-layers this kind on a watch", kind, modes[kind])
|
||||
}
|
||||
}
|
||||
for _, kind := range []string{ToolSettingsKind, ContainerRuntimePoliciesKind, ChannelSettingsKind} {
|
||||
for _, kind := range []string{ToolSettingsKind, ContainerRuntimePoliciesKind} {
|
||||
if modes[kind] != "restart-required" {
|
||||
t.Errorf("%s applies %q, want restart-required: a startup-built component still holds it", kind, modes[kind])
|
||||
}
|
||||
|
||||
@@ -1155,7 +1155,8 @@ type ChatConfig struct {
|
||||
// limiting off entirely -- it is an opt-in control, not a default
|
||||
// throttle that could surprise an existing deployment. This key seeds
|
||||
// the Channel settings resource, which the Web UI owns at runtime
|
||||
// (restart-required): editing the file does not change a stored value.
|
||||
// (live for the channel transports): editing the file does not change a
|
||||
// stored value.
|
||||
RateLimit RateLimitConfig `toml:"rate_limit" yaml:"rate_limit"`
|
||||
}
|
||||
|
||||
@@ -1165,7 +1166,8 @@ type ChatConfig struct {
|
||||
// internal/app/controlplane); the file key is only that resource's seed, and
|
||||
// the Web UI owns the value a running deployment limits with. Applied by
|
||||
// boot.startRateLimiter from the config boot.loadRuntimeConfig layers, so a
|
||||
// stored value takes effect on restart and the resource is restart-required.
|
||||
// stored change to the limit takes effect on the next Gateway restart; the
|
||||
// channel transports are the half the Messaging Service reconciles live.
|
||||
//
|
||||
// A sender is identified per channel: Telegram by numeric user ID, email by the
|
||||
// SMTP from address, webhook by the configured route path -- which is a source,
|
||||
|
||||
Reference in new issue
Block a user