Stands up archie-gateway as its own binary serving ChatContract over gRPC
and makes it the daemon's only transport for webui chat (Q3 part 5's
atomic flip -- the in-process serving path is deleted, not deprecated).
- contract seams + protobuf toolchain (Phase 0)
- single taskactions.Service across webui, daemon NATS responder and the
standalone gateway's client
- services.gateway.target is the only config surface left; mode is gone
now that remote is the only transport
- task dev launches archie-gateway alongside archied + Vite
Verified end-to-end against live infra: real GitHub token (zero repos
configured, so no repo was polled or touched) and the real
@heyarchie_home_bot Telegram bot, both processes booted together against
shared NATS/store, chat and /status round-tripped successfully.
# Conflicts:
# go.sum
Mode had exactly one legal value after the previous commit deleted the
in-process path, which made it dead config surface that could only
confuse an operator into thinking there was still a real choice.
ServiceConnection is now just a target address; composeChatContract only
checks it's non-blank, defaulting to 127.0.0.1:8585.
archie-gateway is now the sole owner of ChatContract; the daemon always
dials it over gRPC for webui chat, no in-process fallback left. Telegram
and email keep their own in-process routers unchanged (their streaming
responder is a callback that can't cross the wire-safe contract).
- composeChatContract only accepts mode="remote"; config defaults
Services.Gateway to remote + 127.0.0.1:8585 when unset
- deleted taskActionsActor (dead code once the local branch is gone)
- task dev now launches archie-gateway alongside archied + Vite
Verified end-to-end: archie-gateway + archied booted against a shared
NATS/store, webui's /api/chat/sessions round-tripped through the gRPC hop.
Phase 1 (extract the Gateway Service), first stage. Repairs the
half-migrated working tree so the default (inproc) daemon path works
again while keeping the WIP's direction:
- chatTaskActorAdapter is contract-only (no HTTP detour); nil contract
returns ErrChatCapabilityUnavailable instead of panicking.
- openStores opens the conversation store for both processes again;
the standalone gateway no longer owns it exclusively. Phase 1 keeps
the daemon's Telegram/email routers and session-memory curator
reading the same SQLite file; remote conversations arrive when a
process that cannot open the file needs them.
- setupLLMAndChat composes the local gateway (setupGatewayChat) in
default mode and a remote gRPC client in mode=remote; the returned
contract is never nil, and Run handles the error.
- taskactions.Service is now the single task-mutation implementation
shared by the dashboard (webui TaskActionService), the daemon's local
gateway actor, and the NATS responder. The daemon registers
taskactions.Register so the standalone Gateway's client can forward
actions; transport follows the repo's natsrpc pattern.
- uncommitted sessions.proto / SessionStoreService gRPC contract
deferred: it has no wired consumers (scope creep for this stage).
- task_action_adapter_test.go rewritten against the contract chain.
task check passes; archied and archie-gateway both build.
* feat(curator): add embedding client and sampler wave-1 prerequisites
Bead archie-core-1786637500752-415-434e5bb9 (GitHub #436) was closed with
no linked PR/commits, and the surprisal-sampling bead
archie-core-1786637500776-416-600a66e5 (GitHub #437) found none of its
stated prerequisites -- an embedding client, a Sampler interface, or the
four wave-1 cheap sampling strategies -- actually exist. This builds them:
- internal/domain/embedding + internal/infrastructure/embedding: a narrow
typed embedding client, config-driven via a providers table + a role in
[models], mirroring the existing chat model role convention. Degrades to
capability-unavailable (never fails startup or a call) for a missing
role, unknown provider, unsupported class, or unresolved credential.
Wired as an optional capability in bootstrap.go's setupEmbeddings.
- internal/domain/sampling: the Sampler interface plus recency, random,
all, and staleness-proxy strategies, per docs/prds/curator-sampler-wave1.md.
Deterministic given fixed inputs, matching GitHub #437's stated
acceptance criterion for the embedding-backed strategy it will add
behind this same interface (not part of this change).
Table-driven tests throughout; httptest for the embedding client, no live
service. No curator wiring yet -- nothing consumes either capability in
this change.
* no-mistakes(document): docs(architecture): mark curator sampling prereqs as landed
The chat agent had no way to know where it lives without probing the
filesystem (pwd, find, git remote), wasting turns on every orientation.
Per docs/architecture/identity.md the <env> block is Layer 1 invariants,
so the daemon's own facts belong there, driven from live configuration.
Render into the prompt's <env> block:
- workspace path (chat.workspace)
- repositories under management with forge host and default branch
- operator identity (chat.operator, which was dead plumbing)
Unconfigured values render explicitly as unknown / none configured
rather than a plausible-looking default, and a repo with no forge
reports forge unknown, so the model never claims a fact the daemon
cannot confirm. Repos are resolved per active chat identity, mirroring
chatTaskProfiles.
gateway: add RepoEnv, SystemPromptConfig Workspace/Repos/Operator,
TurnRunnerConfig Workspace/Repos.
app: chatRepoEnv resolver wired through newChatTurnRunner.
tests: prompt rendering (incl. explicit-unknown + hostile-SOUL guard)
and chatRepoEnv identity resolution. go.sum gains module-graph
/go.mod checksum lines.
Compose the base registry with per-turn extras into a turn-local registry and
let ContextPressureGate decide full-vs-bridge disclosure per chat turn. The
three bridge tools (tool_search, tool_describe, tool_call) are now wired into
the chat toolset and only disclosed under context pressure, keeping the LLM's
schema footprint small while preserving full arsenal access on demand.
Per-turn extras are identity-bound per channel and cannot live in the
process-wide registry (agentexec.BuildToolSetFrom invariant), so the bridge is
composed turn-locally rather than registered globally. Thread the model's
context window through TurnModel.Prepare (TurnPrepareContext) so the gate
activates with a real window.
Refs #167
Implement drain control via an external marker file. The daemon polls
$XDG_CONFIG_HOME/archie/.drain_request.json (the repo's config-home
convention) for a drain request, validates it against the current
instantiation epoch (kernel boot id + PID 1 start time) and, only when
the marker is live, cancels the boot context to trigger the same
graceful shutdown an operator SIGTERM does.
A marker written against a prior instantiation (reboot or PID 1/container
restart) is stale and ignored, so a leftover request never drains a
daemon that came up after it was written. An unreadable marker or epoch
fails closed to no drain. The daemon never invents a new shutdown path.
domain/drain owns the pure decision (Epoch, Marker, Decide);
infrastructure/drain reads the marker file and /proc boot facts;
app/archied wires a poller that cancels the root context on a valid
request.
Path note: the epic named ~/.archie/.drain_request.json, but archied
resolves every per-user path under ${XDG_CONFIG_HOME:-~/.config}/archie
(see AGENTS.md 'Config path'), so the marker lives there too rather than
at a path this daemon never uses.
Refs bead archie-core-1786637496817-266-06b64a72 (drain control: marker
file and instantiation epoch), part of the operations epic
archie-core-1786637496703-262-1e49aef1.
Tests: table-driven Decide (epoch mismatch -> stale, matching -> valid,
nil -> none, unreadable -> stale), marker round-trip from operator JSON,
and monitor (valid -> shutdown, stale/none/error -> no shutdown).
Baseline `task check` failed on a clean main checkout, blocking unrelated
in-flight work. Fixes only those pre-existing failures:
- bootstrap.go: extract wireWebStoreSurfaces from setupObservability
(cyclop 16 > 15) and startEmbeddedNATS from connectNATS (nestif).
- telegram_setup_test.go: os.Setenv -> t.Setenv (usetesting).
- install_script_test.go: exec.Command -> exec.CommandContext with
t.Context() (noctx).
- Taskfile: add a ui:deps install step shared by `ui` and `test:ui`.
ui/node_modules is gitignored and nothing in the gate installed it, so
`task test:ui` died on a missing jsdom rather than a real test failure.
- shutdown_watchdog_test.go: `go fix` output adopted verbatim.
Refs bead archie-core-14cs.
* feat(webui): add /health and /health/detailed readiness endpoints (archie-core-1786637496789-265-b052b772)
Implements the Operations: readiness probes bead from the hermes gap analysis
(docs/prds/hermes-gap-analysis.md sections 9-10).
- GET /health: liveness, no auth, always 200 ok.
- GET /health/detailed: authenticated (dashboard token), JSON report with
overall ok/degraded status plus per-subsystem probe results.
New readiness probes, each reporting real current state:
- state_db: read-only StatusCounts against the store.
- config: configuration.Validate against the running config.
- disk: statfs, degraded above 90% used.
- model: active model configured + provider endpoint reachable.
- gateway: channel running/failed state + connected session count.
Layering: probe contract in internal/domain/health, implementations in
internal/infrastructure/readiness, wiring in internal/app/archied,
HTTP surface in internal/webui.
* no-mistakes(document): docs(archied): fix stale doc pointer in readiness probe comment
Once graceful shutdown begins (SIGINT/SIGTERM cancels the daemon's
signal context), start an out-of-loop watchdog timer for the drain
window plus a 60s grace period. If the shutdown chain has not completed
by the time that leash expires, force-exit the process (os.Exit) as a
backstop against a hung shutdown.
The watchdog runs in its own goroutine outside the daemon's event loop
and graceful-shutdown machinery, so it fires even when that machinery is
itself the thing that is stuck. It is disarmed via the FIRST registered
cleanup so it runs LAST in the LIFO shutdown chain: it stands down only
after every subsystem has shut down cleanly, and a hung subsystem never
reaches the disarm, letting the watchdog force-exit.
A shutdown that completes within the leash does not trigger the
watchdog; a shutdown that hangs past the leash results in a forced
process exit. The leash uses a test-only short timeout in tests.
Reference bead archie-core-1786637496842-267-8b2984f9.
Adds internal/servicediscovery (broker-neutral ServiceRegistry contract,
zero internal imports) and internal/infrastructure/servicediscovery/nats
(NATS JetStream KV implementation: installed-marker bucket + heartbeat
bucket to express NotInstalled distinctly from Down/Unhealthy, per
docs/prds/service-decomposition.md). Additive and unwired -- no daemon
bootstrap path touched. Verified independently: go build ./... clean,
go vet clean, gofmt clean, full package tests pass with -race.
Add a broker-neutral service registry contract (Endpoint, EventKind,
Event, ServiceRegistry, ErrNotInstalled) and a NATS JetStream KV
implementation, plus the registration side a service uses to announce itself.
The contract's governing semantic -- NotInstalled is not Down -- is modeled
by two buckets: a durable installed marker (never expires) distinct from
expiring heartbeat keys. Instances heartbeat their key on an interval; a
stopped process leaves via TTL expiry. Register writes the marker once then
heartbeats; Resolve returns ErrNotInstalled only when the marker is absent,
and an empty roster for installed-but-down. Watch dedupes heartbeat refreshes
so it emits membership changes, not liveness pings.
Unwired: no daemon/bootstrap path consumes it yet. Deliberately uses a dot
key separator instead of the research sketch's slash so per-service watch
wildcards match.
Playbook now carries ID (path-relative to the configured root) and Version
(sha256 content hash recomputed on load), both populated in loadOne, mirroring
Binding.Version's provenance pin without a DB row. DispatchInput gains TaskID
carrying TaskEnvelope.IdempotencyKey(), the identity available at the
discovery point (pollNATS / webhook receiver compute kind+labels from a
TaskEnvelope before any store.Task row exists), NOT a store.Task.ID int64.
These are the two implementation prerequisites named by the t2db.17
idempotency resolution; the playbook_dispatches ledger is built on top.
Note: task check fails on 5 lint findings PRE-EXISTING at HEAD in unrelated
files (bootstrap.go cyclop/nestif, install_script_test.go noctx,
telegram_setup_test.go usetesting), reproduced in a clean worktree and not
caused by this change; the eda/playbook package is gofumpt/vet/lint clean.
/status conflated two unrelated concerns: a thin per-status task
count that duplicated information belonging to a task view, and
runtime config already visible via /model. Worse, a bare status
count actively misled -- "Running: 1" read the same whether that
task was making progress or stuck in a failure loop, because a count
carries no age or identity.
/status now reports one aggregate queue-depth line (in-flight work:
running/waiting/parked, health signal only) instead of an itemized
breakdown -- queued and terminal states are excluded as neither
actionable nor a health concern.
/tasks is new: the live, per-task view with enough identity and
state to act on it (id, title, status, workflow/stage, age since
last transition, attempt once a retry has happened, and a parked
task's reason). Age comes from Task.UpdatedAt, already persisted and
now threaded through ChatTaskSummary and chatTaskListerAdapter --
no new instrumentation needed. Tasks sort actionable-first (running,
waiting, parked, ...) ahead of terminal states, stable within a
status group so the lister's own recency order survives.
Wired via the existing configureTaskCommands call sites (web,
telegram, email, webhook) using the same chatTaskListerAdapter
already built for the task_list LLM tool -- no parallel data path.
Note: internal/gateway's separate /agents command (AgentReader) is
NOT touched here and remains dead in production (Router.Agents is
never wired by any composition-root call site) -- tracked as its own
finding, not folded into this fix.
loadPlaybookFile ranged over the raw YAML map directly, so which key
"won" when a file declared two colliding bindings depended on Go's
randomized map iteration order -- contradicting the function's own
doc comment promise of a reproducible, deterministic collision error.
Surfaced as a flaky failure in TestLintReportsLoadErrorAsFinding
(internal/app/archieplaybooks), which has two independent collisions
in its fixture and asserted on whichever one used to win by chance.
Sort the file's keys before checking them, matching the existing
sorted-filenames convention for cross-file order. Updated the test's
assertion to the now-deterministic outcome ("bug", alphabetically
first) with a comment explaining why.
Real bug found via browser-testing the new bindings UI (t2db.8):
UpdateBinding's own store-layer contract documents that an empty
Secret means "preserve the existing one" (COALESCE against NULLIF),
but handleBindingUpdate validated with the create-time Validate(),
which unconditionally requires a full-length secret -- rejecting the
exact "leave blank to keep it" edit flow the store was built to allow.
Root-cause fix, not a UI workaround: Binding.Validate() (create,
unchanged, still requires a real secret -- there's nothing to fall
back to on insert) now delegates shared field checks to a private
validateCommon(); a new ValidateForUpdate() reuses the same checks but
only enforces the secret-length floor when the caller actually
supplied a new one. handleBindingUpdate now calls ValidateForUpdate.
3 new domain tests + 1 new API-layer test pin: empty secret accepted
on update, a genuinely-short new secret still rejected on update,
empty secret still rejected on create.
Root-cause fix for the correctness gap named in archie-core-t2db.8:
resolveBindingRepo refused to dispatch with zero or multiple configured
repos, meaning a multi-repo deployment could never use bindings at all.
- binding.Binding gains optional Owner/Repo fields; Validate rejects
setting only one (a pin is a complete pair or not a pin at all).
- Persisted as new owner/repo columns (NOT NULL DEFAULT '', additive --
no migration needed since bindings is a CREATE TABLE IF NOT EXISTS
with no existing deployment yet).
- resolveBindingRepo now takes the binding: a pinned binding dispatches
to its own owner/repo unconditionally; an unpinned binding keeps
today's single-configured-repo fallback (0 or 2+ repos still refuses,
unchanged -- verified by a dedicated regression test, not just
assumed preserved).
- Wired through the API request/response shape (create/update accept
and return owner/repo); no repo-existence validation against config
added -- smallest workable change, not the named bug.
Red-green throughout: 8 new tests across domain/store/daemon/webui
layers, each proven to fail before the corresponding implementation
step. Zero regressions in existing binding/dispatch test suites.
Dashboard UI to expose this (the other half of t2db.8) is unstarted --
this closes the backend correctness gap only.
golangci-lint's nilerr finding was syntactically correct but not a bug:
a per-skill parse failure is deliberately converted into a reportable
curator.Action instead of propagated as an execution error, because
Pass()'s loop aborts the whole batch on a non-nil error -- which would
stop reviewing every other skill over one bad file. Confirmed against
skillcurator_test.go's existing "must not abort review of the others"
coverage before touching anything. Scoped //nolint with the reasoning
inline, not a blind suppression.
Seventh slice of docs/prds/eda-playbook-engine.md (t2db.15), consuming
t2db.14's expr package: the rich trigger+actions playbook document and
the coordinator that dispatches it.
- internal/domain/eda/playbook: Load(dir) reads *.yaml/*.yml playbook
documents (trigger + exactly one workflow-kind action + optional CEL
when), validating the HARD BOUNDARY at load: a playbook with 2+ actions
or a non-workflow action position is a reported load failure --
multi-action and Module/Channel/Forge actions stay blocked on the
unresolved execution-time gaps. Any malformed YAML, invalid trigger,
missing workflow name, or when compile failure fails the WHOLE load
(reject-at-load, same shape as routing collisions); a missing dir is an
empty store. Trigger reuses the existing workintake kind/label
vocabulary (no second label-matching mechanism). Dispatch matches
trigger, evaluates when (CEL, event context with actions: {} -- no
prior actions in single-action playbooks), and returns the named
workflow from the registry via the same lookup Route() uses -- no
duplicated dispatch logic. when eval error follows the resolved doc's
J3: false, skip, caller logs.
- Store.Dispatch(reg, DispatchInput) is the coordinator entrypoint; input
carries Labels/Kind/Event. Production task intake wiring
(daemon.go/pollNATS/publishTask) is explicitly NOT in this ticket -- a
follow-up; proven end to end via direct tests plus the shipped example
(examples/eda-playbooks/bug-tdd.yaml) loaded and dispatched through the
real path.
- Config: EDAPlaybookDir (eda_playbook_dir), documented as DISTINCT from
PlaybookDirs (flat binding files) -- a reader cannot confuse the two.
- Bootstrap: playbook.Load wired in loadWorkflows, log-and-abort on
failure matching every prior slice; nonexistent dir is an empty store.
- workflow.Route(), workflowForLabels, and the t2db.9-.12 binding loaders
are UNTOUCHED (verified: their full test suites pass, zero regressions).
Also carries small lint cleanups on t2db.13/.14 code flagged by the gate
run: expr Eval no longer type-asserts to ref.Val (Eval returns ref.Val
directly in v0.32.0; cel-go's own Eval wraps evaluation in recover --
verified in source); module decodeLogArgs extracted to clear the nestif
finding. golangci-lint: 0 issues.
Tests (10): single-action round trip, when-false skip, 2-action load
failure, non-workflow position load failure, when compile failure drops
playbook, empty trigger load failure, trigger mismatch skip, no-when
match, one-invalid-file fails whole load, missing dir empty store, plus
the shipped example load+dispatch. task check clean (113/113 UI, exit 0).
Sixth slice of docs/prds/eda-playbook-engine.md (t2db.14): the
internal/domain/eda/expr package implementing the resolved open question 1
(CEL decision), standalone -- no coordinator/dispatch yet, mirroring how
t2db.13 built ModuleRegistry before any consumer.
- NewEnv: CEL env declaring the two context roots (event + actions, both
map(string, dyn)) with DefaultCostLimit (100,000, tunable per J5).
- Compile: parses/type-checks an expression string; syntax error, unknown
root (anything other than event/actions), or type error is a returned
error -- the reject-at-load entry point the playbook loader and lint
tool will share.
- Eval: evaluates a compiled program against a dispatch-time Context;
missing-field/wrong-type/cost-limit errors are returned, never panics.
No recover() wrapper: CEL is panic-free by design, verified by the
hostile-data tests (5000-deep map, no panic). CEL's ref.Val wrappers
(types.Bool etc.) are unwrapped to native Go values.
Dependency: cel.dev/cel-go v0.32.0 (direct), the only new dependency,
already vetted in the resolved doc's vetting record -- no re-vet.
Tests (table-driven, verified against real CEL semantics, not guessed):
valid boolean against event fields; syntax error at compile; unknown root
rejected at compile naming the root; missing event field errors not
panics (== on a missing key evaluates false -- verified CEL semantics);
has() presence macro; actions context reads; deep-nested hostile data
error-free or false without panic; cost-limit enforcement proven by
aborting a macro expression under a tiny limit (CEL's cost model accrues
on map/filter iteration, not field reads -- verified by probing).
task check clean (113/113 UI, exit 0).
Fifth slice of docs/prds/eda-playbook-engine.md (t2db.13), implementing
docs/prds/module-position.md's recommended first slice: prove schema ->
go:generate -> Yaegi load -> ModuleRegistry end to end with a
side-effect-free action kind, before any kind with a real external effect
(notify, image-gen, forge writes) -- those stay blocked on the parent
doc's unresolved execution-time gaps (idempotency, mid-run failure).
- internal/domain/eda/module/log: hand-written schema source (Args/Result/
SchemaVersion per module-position.md) + logextract generated symbol
table via go:generate yaegi extract (same tooling as wfextract/
gateextract; scoped to ./internal/domain/eda/module/...).
- internal/domain/eda/module: ModuleRegistry with Register(kind, dir) /
Invoke(ctx, kind, rawArgs). No generic Module interface with an any
payload: each kind is its own tiny package with its own generated
contract; the registry's internal storage is the only place erasure
appears (exactly as inside wfextract). Invoke decodes rawArgs strictly
into the kind's Args before calling the interpreted function -- an
unknown key or wrong type is a reported failure, not a silent
zero-value fill (the 'schema defines the accepted message' rule).
Generalizes gateeval's fixed-signature Yaegi pattern via
yaegiutil.New/Resolve/Safe -- no new interpreter-construction or
panic-recovery mechanism.
- Config: ModuleDir mirroring PluginDir's shape and doc comment
(operator-trusted directory of Yaegi module implementations, one .go
per kind; empty means none loaded). Wired at daemon startup in
bootstrap.go alongside PlaybookDirs loading -- log-and-abort on load
failure (a broken module is a startup failure, not degrade-and-skip).
- examples/modules/log.go: operator-facing example (//go:build ignore,
mirroring examples/secret-engines/) proving the shipped contract.
Tests: registry (register/invoke, unknown kind, missing file, strict
arg-decode shape failures, malformed file, panic recovery), logextract
smoke test, and an example-load test that runs the shipped
examples/modules/log.go through the real registry (not a test fixture).
task check clean (113/113 UI, exit 0).
Playbook YAML action dispatch is deliberately out of this slice: a
minimal 'run this one module action, no chaining' dispatch still requires
naming how args values reach the action (parent doc open question 1 --
data-flow/interpolation syntax) and what the result feeds, both
unresolved by design. Dispatch is a follow-up ticket that consumes this
registry as-is; recorded in module-position.md.
Two pre-existing issues discovered while scoping the Module-position slice
(archie-core-e7bo), unrelated to the playbook engine work:
1. wfextract's //go:generate directive still targeted
internal/workflow, a path gone since the domain migration -- the
committed generated file was hand-patched for imports only, never
regenerated. 'go generate ./...' failed repo-wide, silently.
2. Regeneration surfaced real drift on three surfaces:
- wfextract: the committed table predated the migration and was
missing everything added since -- routing YAML loaders (t2db.9/.10/
.11), the review types, and the Forger/Reviewer/Trees interfaces
that TaskContext exposes to every interpreted stage plugin. A stage
plugin referencing tc.Trees or the review types could not resolve
them.
- pluginextract: internal/plugin/host.go (f73b2d1) added the
Host/Module/Manifest/Permission/Health/CapabilityKind/LifecycleState
/ModuleStatus types and NewHost/AdaptLegacy/HostAPIVersion, none of
which were ever extracted. A daemon plugin written against the
capability-host API (bootstrap.go:707 LoadDir) silently had no way
to reference them.
- secretextract: symbol exports were current; only the hand-applied
nil-guards and comments differ from raw generator output.
Fix: update the wfextract directive to internal/domain/workflow and
regenerate all four extract surfaces via 'go generate ./...'. Re-apply
the documented hand-written nil-guards on interface wrappers (Yaegi
wrappers omit them natively), restating the IMPORTANT note in wfextract's
package doc (it now has interface wrappers for the first time). Delete
the stale hand-patched wfextract generated file in favour of the
correctly-named regeneration.
Regression coverage (the previously-missing types are now reachable from
interpreted code, proven by tests that fail against the old tables):
- wfextract/reachability_test.go: interprets a stage plugin referencing
Trees/Reviewer/LoadPlaybookDirs and calls it; plus a Go-side nil-guard
test on the three wrappers.
- pluginextract/reachability_test.go: interprets a daemon plugin building
a Host from a Go-side Module and calls it end to end; plus nil-guard
tests on the _Module/_Plugin wrappers.
- secretextract keeps its existing wrapper nil-guard tests.
Verified red-then-green: after a fresh 'go generate ./...' wipes the
guards, the guard tests fail with nil-pointer panics; after re-applying,
they pass. task check clean (113/113 UI, exit 0).
Fourth slice of docs/prds/eda-playbook-engine.md (t2db.12): the design
doc's pre-merge collision layer, as a gopls-shaped standalone binary
(cmd/archie-playbooks) rather than a Taskfile wrapper -- one binary, one
schema source, mode-based entrypoints.
- cmd/archie-playbooks: subcommand dispatch table from day one (lint now;
an lsp/serve mode later slots in without restructuring), flag parsing
only per organisation.md's cmd/ rule.
- internal/app/archieplaybooks: the lint mode's composition -- collects
directories, calls workflow.LoadPlaybookDirs (the daemon's own loader),
reports findings. No second validation path: the linter agrees with
runtime startup validation by construction.
- Reuses LoadKindWorkflowsYAML/LoadLabelWorkflowsYAML/LoadPlaybookDirs
directly; collision/malformed/invalid bindings exit non-zero.
- Known limitation recorded: findings are file-granular, not
line-granular (yaml.Unmarshal into map discards line numbers; a
file:line diagnostic needs a yaml.Node upgrade, out of scope here).
- Taskfile lint:playbooks target; PRD records the shipped shape and the
file-granular caveat.
Third slice of docs/prds/eda-playbook-engine.md (t2db.11), completing open
question 2: instead of a single playbook dir, the config gains a LIST of
directories (playbook_dirs) of *.yaml/*.yml binding files, loaded at
startup as an additional input to workflow_routing_file and
workflow_labels_file, which remain supported unchanged.
- LoadPlaybookDirs(dirs []string) reads every yaml file across the
configured directories and merges kind + arbitrary-label bindings.
- This is what finally exercises the design doc's drop-and-report
collision rule: a single file cannot collide with itself, so the
cross-source path (two files in one dir, two dirs, or a dir plus a
single-file field) had never run in production. A binding key declared
by more than one source is a reported load failure, not a silent
pick-one by source order or version.
- Router-level collision handling: MergeKindWorkflows/MergeLabelWorkflows
fail on a key bound by both the single-file fields and the playbook
dirs, applied at boot.
- Config: PlaybookDirs []string, empty by default (no behavior change for
existing configs). Repo-scoping (typing dirs to Config.Repos owner/name)
is explicitly deferred to a separate later decision per ticket;
org/tenant keying is out of scope.
- PRD: records the list shape per Sam's correction (single dir would
contradict multiple independently-maintained playbook sources).
Second slice of docs/prds/eda-playbook-engine.md, complementing the
kind->workflow layer (WorkflowRoutingFile): the label vocabulary itself
becomes playbook data. A label outside the closed bug/feature/bootstrap
kind set (e.g. security) now dispatches to a registered workflow from an
optional WorkflowLabelsFile, instead of falling through to triage.
- LabelWorkflows + LoadLabelWorkflowsYAML/SetLabelWorkflows in
internal/domain/workflow/routing.go, mirroring the kind layer's
once-at-startup lifecycle. The closed Kind/NATS-subject set is
untouched; the map only extends binding authority for labels the kind
layer does not own.
- Collision rule per design doc: a label owned by the kind set, an empty
label, an empty workflow name, or a duplicate binding is a reported
load failure (dropped, logged, error returned to the caller) -- the
schema defines what is accepted; nothing is silently arbitrated by
schema/version.
- Route() precedence: explicit t.Workflow -> arbitrary-label binding ->
kind binding -> triage -> implement -> default.
- Config: WorkflowLabelsFile (workflow_labels_file), wired at boot
alongside WorkflowRoutingFile; documented in config.example.toml.
- PRD: records this slice's decisions (single-file shape, collision rule,
precedence) so later slices stay anchored.
First slice of docs/prds/eda-playbook-engine.md: which registered
workflow an intake Kind (bug/feature/bootstrap) prefers is now data,
loaded from an optional WorkflowRoutingFile, instead of a Go literal.
labelKinds and the NATS-subject-per-kind set are unchanged -- expanding
the label vocabulary itself is a separate, larger decision (subject
allocation) deferred per the design doc's open questions.
An unrecognised kind in the file is a reported load failure (schema
defines what's accepted, per the design doc's collision-handling rule),
not a silently-dropped binding. No file configured falls back to
today's hardcoded defaults unchanged.
append(nonce, sealed...) appends to a slice initialized with a non-zero
length, which makezero flags. Build the nonce||ciphertext||tag payload with
an explicit buffer sized upfront instead.
Found by golangci-lint.
RedactPayload only redacted values under a fixed set of sensitive key
names, so a sender that names a secret field something else leaked it into
captured_events.body. Harden the heuristic:
- broaden the key-name markers (compound names like access_token,
client_secret, signing_key, x_api_key, webhook_secret, refresh_token)
and keep gateway.sensitiveParameterKey identical to preserve the
documented agreement between the two redaction paths.
- add value-shape detection: a compact JWT or a PEM private-key block is
redacted under any key, catching the 'sender named the field something
the heuristic misses' gap.
- keep it conservative so schema-by-example field mapping still works:
only unambiguous shapes (eyJ-prefixed JWTs, PEM key blocks) are
redacted by value, never ordinary content.
Best-effort per docs/prds/webhook-intake-security.md point 5.
Refs archie-core-t2db (captured_events.body redaction)
binding.Binding.Secret was persisted as plaintext in the bindings.secret
column, so anyone with the SQLite file saw sender-side HMAC secrets. Seal
it with AES-256-GCM transparently inside the store: encrypt-on-write,
decrypt-on-read.
- cipher: AES-256-GCM, 12-byte random nonce, SHA-256-normalised 32-byte
key resolved through the secret registry ([bindings].encryption_key).
- envelope 'arcie-binding:v1:<fingerprint>:<b64(nonce||ct||tag)>' so a
future cipher/KDF change is a version bump and old rows stay readable.
- rotation via previous_encryption_keys; key identity is the material's
SHA-256 fingerprint embedded in each row.
- legacy plaintext rows written before a key was configured stay readable;
encryption is not retroactive.
- nil cipher (no key configured) preserves the pre-t2db.7 plaintext path.
Deferred: captured_events.body redaction upgrade (separate concern, see
docs/prds/binding-secret-encryption.md).
Refs archie-core-t2db.7
Implements archie-core-t2db.4: the runtime-editable binding entity that
turns a captured webhook into an archie task, plus the per-source HMAC
verification at handleCapture that makes t2db.5's 'unauthenticated event
cannot trigger a binding' invariant structurally enforceable.
Captures, mappings and bindings are now separate concerns on the same
single *store.Store; the new BindingStore / BindingDispatcher /
BindingTaskCreator split keeps each interface under the interfacebloat
limit (8 methods).
State machine: draft -> pending_approval -> armed; any edit on an armed
binding drops it back to pending_approval, enforced in SQL via
status='pending_approval' on UPDATE so the rule cannot be bypassed by a
direct status field in the request body. Approve is the only sanctioned
transition into armed (WHERE status='pending_approval' guard).
Overlap: two bindings may not share a source in any status; enforced at
InsertBinding and UpdateBinding (transaction-local SELECT). Approve has
a separate TOCTOU guard so a race between Insert and Approve cannot
// bypass the check.
HMAC at handleCapture: reads X-Hub-Signature-256 (then X-Signature-256)
// fallback, verifies via webhookguard.VerifyHMAC against the binding's
// per-source secret, marks CapturedEvent.Authenticated=true on match.
// Captures with no armed binding, missing signature, or wrong signature
all persist as authenticated=false -- the dispatch loop's
binding.Matcher.Matches gate (requires authenticated) is what stops
them from spawning a task. Two armed bindings for one source returns
409 at capture time as a belt-and-braces TOCTOU guard.
Dispatch loop: added Daemon.dispatchBindings, called from both Cycle
and maintainAndDrain between poll and drain. Walks
captured_events where authenticated=1 AND source IN armed bindings
AND id NOT IN binding_dispatches, runs the binding's matcher, resolves
the mapping, enqueues a chat-style task with binding_id /
// binding_version provenance stamped in the same transaction as the
// row insert. The binding_dispatches ledger (composite PK on
(binding_id, capture_id)) is the at-most-once guarantee across cycles
// and daemon restarts.
Tasks schema migration: binding_id INTEGER NOT NULL DEFAULT 0 +
binding_version INTEGER NOT NULL DEFAULT 0 (taskSchemaVersion bumped
1 -> 2). Existing tasks keep 0 / 0.
Out of scope (documented follow-ups):
- secret encryption at rest (plaintext today)
- full dashboard HTML for the binding editor (CRUD API only this bead)
- per-binding rate limits (existing per-remote-address limit applies)
- binding.Owner / binding.Repo fields (dispatch falls back to the
single configured repo; multi-repo deployments log and skip)
Adds PATCH /api/config/repos/{owner}/{name}, a dedicated seam distinct
from the generic dotted-key PATCH /api/config path: a repository is
one element of []config.Repo, not a flat key. Traced the mechanism
before adding any new plumbing -- overlay.Nest/ApplyOverlayValues
already handle "repos" as a single whole-slice key with no dot, so
the fix is a webui handler plus a bootstrap seam that reads the
daemon's live, untrimmed Repos (never webui.RepoView, which omits
Preflight/TestGlob), mutates the one matched field via the pure
applyRepoFieldUpdate helper, and republishes through the existing
UpdateConfig(ctx, {"repos": repos}) path unchanged.
config-row.js's row() gains an opts.save override so an edit can be
redirected to a different persistence call. The repositories table
renders an autosaving checkbox for the two booleans and a
save-on-blur/Enter input for max_retries, via api.configRepoUpdate.
Closes archie-core-b6ew.6, and with it archie-core-b6ew.