Commit Graph
77 Commits
Author SHA1 Message Date
sam c6be68816e feat(eda): playbook document + event coordinator, single-action workflow-kind dispatch
Seventh slice of docs/prds/eda-playbook-engine.md (t2db.15), consuming
t2db.14's expr package: the rich trigger+actions playbook document and
the coordinator that dispatches it.

- internal/domain/eda/playbook: Load(dir) reads *.yaml/*.yml playbook
  documents (trigger + exactly one workflow-kind action + optional CEL
  when), validating the HARD BOUNDARY at load: a playbook with 2+ actions
  or a non-workflow action position is a reported load failure --
  multi-action and Module/Channel/Forge actions stay blocked on the
  unresolved execution-time gaps. Any malformed YAML, invalid trigger,
  missing workflow name, or when compile failure fails the WHOLE load
  (reject-at-load, same shape as routing collisions); a missing dir is an
  empty store. Trigger reuses the existing workintake kind/label
  vocabulary (no second label-matching mechanism). Dispatch matches
  trigger, evaluates when (CEL, event context with actions: {} -- no
  prior actions in single-action playbooks), and returns the named
  workflow from the registry via the same lookup Route() uses -- no
  duplicated dispatch logic. when eval error follows the resolved doc's
  J3: false, skip, caller logs.
- Store.Dispatch(reg, DispatchInput) is the coordinator entrypoint; input
  carries Labels/Kind/Event. Production task intake wiring
  (daemon.go/pollNATS/publishTask) is explicitly NOT in this ticket -- a
  follow-up; proven end to end via direct tests plus the shipped example
  (examples/eda-playbooks/bug-tdd.yaml) loaded and dispatched through the
  real path.
- Config: EDAPlaybookDir (eda_playbook_dir), documented as DISTINCT from
  PlaybookDirs (flat binding files) -- a reader cannot confuse the two.
- Bootstrap: playbook.Load wired in loadWorkflows, log-and-abort on
  failure matching every prior slice; nonexistent dir is an empty store.
- workflow.Route(), workflowForLabels, and the t2db.9-.12 binding loaders
  are UNTOUCHED (verified: their full test suites pass, zero regressions).

Also carries small lint cleanups on t2db.13/.14 code flagged by the gate
run: expr Eval no longer type-asserts to ref.Val (Eval returns ref.Val
directly in v0.32.0; cel-go's own Eval wraps evaluation in recover --
verified in source); module decodeLogArgs extracted to clear the nestif
finding. golangci-lint: 0 issues.

Tests (10): single-action round trip, when-false skip, 2-action load
failure, non-workflow position load failure, when compile failure drops
playbook, empty trigger load failure, trigger mismatch skip, no-when
match, one-invalid-file fails whole load, missing dir empty store, plus
the shipped example load+dispatch. task check clean (113/113 UI, exit 0).
2026-09-03 04:05:59 +10:00
sam 54e3b49fa7 docs(prds): resolve data-flow/condition syntax as CEL (open question 1)
CEL (cel.dev/cel-go, pinned v0.32.0) is the single expression mechanism
covering both an action's `when` condition and its `args` values,
evaluated against a read-only context (`event`, `actions.<id>.result`).
Rejects the custom-gate precedent (Yaegi-in-YAML) as inverting the
design's own "YAML is data, Yaegi is logic" premise. Expressions
compile/type-check at playbook load, consistent with the existing
collision-handling reject-at-load rule; generated Module Result schemas
become CEL type declarations so result-path typos reject at load too.

Vetted against expr-lang/expr, gval, govaluate (archived), starlark-go:
CEL wins on structural safety (non-Turing-complete, CostLimit-bounded,
error values not panics, zero security advisories on record) despite
the largest binary cost (+13MB) of the candidates -- accepted because
the alternative is re-implementing CEL's type-checker on a bespoke
grammar for the same guarantee.

Sign-off (Sam, 2026-09-03): all judgment calls (J1-J5) accepted as
proposed, including J2's per-value CEL-literal quoting cost and the
+13MB binary size trade.

Cross-references in module-position.md updated to point at this
resolution instead of citing open question 1 as unresolved.
2026-09-03 03:55:24 +10:00
sam 7ef7d696f3 feat(eda): Module position first slice -- registry + side-effect-free log kind
Fifth slice of docs/prds/eda-playbook-engine.md (t2db.13), implementing
docs/prds/module-position.md's recommended first slice: prove schema ->
go:generate -> Yaegi load -> ModuleRegistry end to end with a
side-effect-free action kind, before any kind with a real external effect
(notify, image-gen, forge writes) -- those stay blocked on the parent
doc's unresolved execution-time gaps (idempotency, mid-run failure).

- internal/domain/eda/module/log: hand-written schema source (Args/Result/
  SchemaVersion per module-position.md) + logextract generated symbol
  table via go:generate yaegi extract (same tooling as wfextract/
  gateextract; scoped to ./internal/domain/eda/module/...).
- internal/domain/eda/module: ModuleRegistry with Register(kind, dir) /
  Invoke(ctx, kind, rawArgs). No generic Module interface with an any
  payload: each kind is its own tiny package with its own generated
  contract; the registry's internal storage is the only place erasure
  appears (exactly as inside wfextract). Invoke decodes rawArgs strictly
  into the kind's Args before calling the interpreted function -- an
  unknown key or wrong type is a reported failure, not a silent
  zero-value fill (the 'schema defines the accepted message' rule).
  Generalizes gateeval's fixed-signature Yaegi pattern via
  yaegiutil.New/Resolve/Safe -- no new interpreter-construction or
  panic-recovery mechanism.
- Config: ModuleDir mirroring PluginDir's shape and doc comment
  (operator-trusted directory of Yaegi module implementations, one .go
  per kind; empty means none loaded). Wired at daemon startup in
  bootstrap.go alongside PlaybookDirs loading -- log-and-abort on load
  failure (a broken module is a startup failure, not degrade-and-skip).
- examples/modules/log.go: operator-facing example (//go:build ignore,
  mirroring examples/secret-engines/) proving the shipped contract.

Tests: registry (register/invoke, unknown kind, missing file, strict
arg-decode shape failures, malformed file, panic recovery), logextract
smoke test, and an example-load test that runs the shipped
examples/modules/log.go through the real registry (not a test fixture).
task check clean (113/113 UI, exit 0).

Playbook YAML action dispatch is deliberately out of this slice: a
minimal 'run this one module action, no chaining' dispatch still requires
naming how args values reach the action (parent doc open question 1 --
data-flow/interpolation syntax) and what the result feeds, both
unresolved by design. Dispatch is a follow-up ticket that consumes this
registry as-is; recorded in module-position.md.
2026-09-03 03:35:18 +10:00
sam 319ec85977 docs(prds): add Module position design doc
Resolves eda-playbook-engine.md's deferred schema-generation question
for the Module action position. Generalizes the fixed-signature,
resolved-by-name Yaegi pattern already shipped three times (custom
gate, workflow stage plugins, skill plugins) instead of inventing a
new mechanism: one small hand-written schema package per action kind,
go:generate yaegi extract (the same command wfextract/gateextract
already use) producing its symbol table, a ModuleRegistry keyed by
kind rather than a generic any-typed Module interface -- keeping every
position individually compliant with the plugin engine rule instead
of reopening the generic-hook question event-sources-and-reactions.md
already closed.

Closes open question 3 (trust boundary: operator-installed, in-process,
same tier as PluginDir/SecretEngineDir -- not repository-supplied task
code). Recommends a side-effect-free first Module kind, since any kind
with a real external effect is still blocked on the parent doc's
unresolved execution-time gaps (idempotency, mid-run failure).
2026-09-03 03:10:09 +10:00
sam c39d7b8040 feat(playbooks): standalone lint tool for playbook binding dirs
Fourth slice of docs/prds/eda-playbook-engine.md (t2db.12): the design
doc's pre-merge collision layer, as a gopls-shaped standalone binary
(cmd/archie-playbooks) rather than a Taskfile wrapper -- one binary, one
schema source, mode-based entrypoints.

- cmd/archie-playbooks: subcommand dispatch table from day one (lint now;
  an lsp/serve mode later slots in without restructuring), flag parsing
  only per organisation.md's cmd/ rule.
- internal/app/archieplaybooks: the lint mode's composition -- collects
  directories, calls workflow.LoadPlaybookDirs (the daemon's own loader),
  reports findings. No second validation path: the linter agrees with
  runtime startup validation by construction.
- Reuses LoadKindWorkflowsYAML/LoadLabelWorkflowsYAML/LoadPlaybookDirs
  directly; collision/malformed/invalid bindings exit non-zero.
- Known limitation recorded: findings are file-granular, not
  line-granular (yaml.Unmarshal into map discards line numbers; a
  file:line diagnostic needs a yaml.Node upgrade, out of scope here).
- Taskfile lint:playbooks target; PRD records the shipped shape and the
  file-granular caveat.
2026-09-03 03:07:27 +10:00
sam c1f92946f2 docs(prds): record shared-package rationale for the playbook lint/LSP tool
Not a bundled-vs-standalone tradeoff: validation logic lives in
internal/domain/workflow with no knowledge of its caller, so archied
and agents already validate in-process, and the standalone cmd/
binary (t2db.12) exists only because CI/editors are out-of-process
consumers by nature. One implementation, two thin consumer shapes.
2026-09-03 03:03:06 +10:00
sam dc951d5601 feat(workflow): load kind/label bindings from multiple playbook directories
Third slice of docs/prds/eda-playbook-engine.md (t2db.11), completing open
question 2: instead of a single playbook dir, the config gains a LIST of
directories (playbook_dirs) of *.yaml/*.yml binding files, loaded at
startup as an additional input to workflow_routing_file and
workflow_labels_file, which remain supported unchanged.

- LoadPlaybookDirs(dirs []string) reads every yaml file across the
  configured directories and merges kind + arbitrary-label bindings.
- This is what finally exercises the design doc's drop-and-report
  collision rule: a single file cannot collide with itself, so the
  cross-source path (two files in one dir, two dirs, or a dir plus a
  single-file field) had never run in production. A binding key declared
  by more than one source is a reported load failure, not a silent
  pick-one by source order or version.
- Router-level collision handling: MergeKindWorkflows/MergeLabelWorkflows
  fail on a key bound by both the single-file fields and the playbook
  dirs, applied at boot.
- Config: PlaybookDirs []string, empty by default (no behavior change for
  existing configs). Repo-scoping (typing dirs to Config.Repos owner/name)
  is explicitly deferred to a separate later decision per ticket;
  org/tenant keying is out of scope.
- PRD: records the list shape per Sam's correction (single dir would
  contradict multiple independently-maintained playbook sources).
2026-09-03 02:57:38 +10:00
sam 06f4bbe3b3 feat(workflow): route arbitrary forge labels to workflows via playbook YAML
Second slice of docs/prds/eda-playbook-engine.md, complementing the
kind->workflow layer (WorkflowRoutingFile): the label vocabulary itself
becomes playbook data. A label outside the closed bug/feature/bootstrap
kind set (e.g. security) now dispatches to a registered workflow from an
optional WorkflowLabelsFile, instead of falling through to triage.

- LabelWorkflows + LoadLabelWorkflowsYAML/SetLabelWorkflows in
  internal/domain/workflow/routing.go, mirroring the kind layer's
  once-at-startup lifecycle. The closed Kind/NATS-subject set is
  untouched; the map only extends binding authority for labels the kind
  layer does not own.
- Collision rule per design doc: a label owned by the kind set, an empty
  label, an empty workflow name, or a duplicate binding is a reported
  load failure (dropped, logged, error returned to the caller) -- the
  schema defines what is accepted; nothing is silently arbitrated by
  schema/version.
- Route() precedence: explicit t.Workflow -> arbitrary-label binding ->
  kind binding -> triage -> implement -> default.
- Config: WorkflowLabelsFile (workflow_labels_file), wired at boot
  alongside WorkflowRoutingFile; documented in config.example.toml.
- PRD: records this slice's decisions (single-file shape, collision rule,
  precedence) so later slices stay anchored.
2026-09-03 02:44:41 +10:00
sam 4acfa28966 docs(prds): add EDA playbook engine design doc
Records the design for drop-in, data-driven workflows/labels/actions:
YAML playbooks as pure orchestration data over four typed logic
positions (Workflow/Module/Channel/Forge), go:generate-driven schemas,
and load-time collision handling (detect+log+report, no auto-arbitration).
Flags execution-time idempotency and mid-run failure handling as
unresolved gaps to close before Channel/Forge or multi-action playbooks
ship.
2026-09-03 02:22:04 +10:00
sam f9f9888702 feat(webhook): harden captured payload redaction
RedactPayload only redacted values under a fixed set of sensitive key
names, so a sender that names a secret field something else leaked it into
captured_events.body. Harden the heuristic:

- broaden the key-name markers (compound names like access_token,
  client_secret, signing_key, x_api_key, webhook_secret, refresh_token)
  and keep gateway.sensitiveParameterKey identical to preserve the
  documented agreement between the two redaction paths.
- add value-shape detection: a compact JWT or a PEM private-key block is
  redacted under any key, catching the 'sender named the field something
  the heuristic misses' gap.
- keep it conservative so schema-by-example field mapping still works:
  only unambiguous shapes (eyJ-prefixed JWTs, PEM key blocks) are
  redacted by value, never ordinary content.

Best-effort per docs/prds/webhook-intake-security.md point 5.

Refs archie-core-t2db (captured_events.body redaction)
2026-09-02 23:59:43 +10:00
sam 087f161836 feat(store): encrypt playbook binding secrets at rest
binding.Binding.Secret was persisted as plaintext in the bindings.secret
column, so anyone with the SQLite file saw sender-side HMAC secrets. Seal
it with AES-256-GCM transparently inside the store: encrypt-on-write,
decrypt-on-read.

- cipher: AES-256-GCM, 12-byte random nonce, SHA-256-normalised 32-byte
  key resolved through the secret registry ([bindings].encryption_key).
- envelope 'arcie-binding:v1:<fingerprint>:<b64(nonce||ct||tag)>' so a
  future cipher/KDF change is a version bump and old rows stay readable.
- rotation via previous_encryption_keys; key identity is the material's
  SHA-256 fingerprint embedded in each row.
- legacy plaintext rows written before a key was configured stay readable;
  encryption is not retroactive.
- nil cipher (no key configured) preserves the pre-t2db.7 plaintext path.

Deferred: captured_events.body redaction upgrade (separate concern, see
docs/prds/binding-secret-encryption.md).

Refs archie-core-t2db.7
2026-09-02 23:54:06 +10:00
sam 2980ffa7c7 docs(webui): design a backend-owned config schema for the Configuration page
Field metadata (label, section, type, editability, restart-required)
currently lives only in settings.js/config-row.js, disjoint from the
backend's ConfigView allowlist -- adding or changing a runtime-tunable
setting means editing Go and JS by hand in two places with nothing to
catch drift. Repo.ReviewEnabled never reaches the API at all, and
AllowConcurrent/MaxRetries reach it but the repositories card never
renders them.

Decision: keep the existing secret-free ConfigView allowlist as the
safety boundary (no reflection over config.Config, which is already
scheduled for dissolution), and hand-author a descriptor per allowed
field so the frontend renders sections/fields generically instead of
hardcoding rows.

Filed as archie-core-b6ew (epic) with five dependency-ordered children.
2026-09-02 22:01:41 +10:00
sam dea69d895d fix(gateway): stop SOUL from forging the prompt invariant tags
Persona/SOUL text was interpolated raw at the top of archie.md.tpl,
so a hostile SOUL could inject its own <core_rules> or
<instruction_precedence> tags ahead of the real ones. Render it
through the existing xml-escaping helper into a labeled
trust="data" <soul> block, and tell instruction_precedence
explicitly that the block is reference data, not an authority.

Closes archie-core-1786637490097-12-bf415ffb
2026-09-02 21:48:57 +10:00
sam 5bec08b6ce feat(workflow): dynamic triage instead of defaulting every task to implement
Route() had no content-aware fallback: any task with no explicit
workflow and no label match ran the full "implement" pipeline
regardless of content. Confirmed in production (task 6): a
chat-spawned task whose entire content was "this is just a test,
close it when you receive it" burned 669,421 tokens finding that out
the expensive way -- baseline, plan, and build all ran before the
plan stage's own conclusion (nothing needed to change) arrived too
late to save anything.

Adds a "triage" workflow: one cheap, read-only classification turn
(mirrors feasibility.go's decide-tool pattern) that either closes a
task needing no code change immediately, or sets Task.Workflow and
requeues under whichever workflow fits. Route()'s fallback now
prefers a registered "triage" over "implement", unchanged when
triage isn't registered. Labeled tasks are untouched -- a label is
already a free, reliable signal, so triage never fires there.

See docs/prds/dynamic-workflow-triage.md for the full design.
2026-09-02 21:40:06 +10:00
sam 45b183b325 docs(image): point the hosted-provider bead at ai-sdk's existing image facility
ai-sdk (already a dependency) has a provider-agnostic image.Provider,
core.GenerateImage, and registry.RegisterImage -- three backends
(azure/togetherai/xai) implement it, OpenAI doesn't yet. The next
bead should adapt that instead of a raw HTTP client, mirroring how
curatorLLMRunner adapts runtime.Runtime for chat. Noted the one real
gap (no edit/mask support in ai-sdk's request type) for that bead to
resolve at its own design step.
2026-09-02 20:23:14 +10:00
sam 75158b7fc7 feat(image): define provider-neutral image capability contract and config
Adds internal/domain/image (Provider/Registry per the plugin engine
family shape, but no Lifecycle -- a provider is call-scoped, not
long-running) and config.ImageConfig with the same explicit
enabled=true cost gate MinimaxConfig already uses, since a hosted
provider spends real API credits and a local one can saturate the
host GPU. Design doc at docs/prds/image-capability-contract.md per
CLAUDE.md's rule for undesigned capabilities.

No provider implementation lands here -- that's the epic's next
child bead. Also re-links all 7 epic children that were unlinked
from archie-core-1786748942120-1-6636e629 (same orphaning pattern
found in the curator/memory epics earlier this session).
2026-09-02 20:20:54 +10:00
sam fc6bffad50 feat(curator): session-memory curator reference implementation
Closes archie-core-1786637499114 (persistent session-memory curator),
the last open item in the curator engine epic.

docs/prds/session-memory-curator.md records the decisions this needed
before coding, most notably: Observation.Identity is the session ID,
not a user or cross-session identity. migration-decisions.md#5's
memory-scope mapping (global/agent-wide/user-wide/agent-user
relationship) is still explicitly open, and this issue has no business
deciding it unilaterally -- session-as-identity is the narrowest
unambiguous scope available today and is forward-compatible with a
later, properly-decided identity resolution.

curator gains ConversationSource (RecentSessions/Messages), gated in
Registry.validateDeclared/filter exactly like Skills/MemoryEngines, plus
Manifest.Conversations. internal/infrastructure/sessioncurator/adapter.go
adapts gateway.SessionStore's full CRUD/branch/search surface down to
those two read-only methods -- curator must not import gateway.Message,
so role derivation (message.From == botUser -> assistant) is copied from
gateway.compressTurnHistory's existing logic, not shared.

sessioncurator.go: Check is a cheap RecentSessions bool. Pass takes each
session's last 40 messages, asks the model for a JSON array of durable
facts (capped at 5/session), and writes each through
Registrar.MemoryEngines.Get(name) -- resolved by name at call time, the
same trust model every other curator capability already uses. A
response that fails to parse degrades to "nothing extracted" for that
session (not a Pass error); a real model-call failure propagates as one
(these were originally conflated into the same swallowed-error path --
caught by TestPassPropagatesALLMFailure, fixed by splitting the model
call from the parse step). PassInput.Since zero resolves to a 24h
lookback, not the beginning of time, so a cold-start first pass can't
read a session's entire history.

Manifest.OnInput = true wakes it via the existing WakeOnPrimaryInput
wiring (events.KindTurnCompleted), which already excludes every
curator-derived event kind from the forwarded set -- the trigger-
accounting invariant (archie-core-1786637489768) needed no new code to
hold here.

Wired into bootstrap.go:setupCurators via a new curatorLLMRunner adapter
(main.go) over the shared ai-sdk runtime, and sessioncurator.NewAdapter
over b.chatSessionStore/b.cfg.BotUser. Model resolves from
b.chatModels.ActiveModel() -- not b.defaultChatIdentity, which names a
task-routing identity, not a model reference (the same source
telegram_setup.go's sendChatTurn already uses for a real chat turn).

task check is green (full Go suite + dashboard tests).
2026-09-02 03:09:04 +10:00
sam 8164d21348 feat(curator): skill curator reference implementation
Part of archie-core-1786637499206 (skill curator reference
implementation), under the curator engine epic.

docs/prds/skill-curator.md records the design decision this needed
before coding (no settled design existed for what "review, prune, and
improve skill definitions" means as concrete Pass() behavior): v1 is
mechanical validation and safe whitespace normalization only. No
LLM-authored rewrites of skill content, no automatic deletion -- an
autonomous process editing or deleting a user's skill has no undo path
today, and the epic's actual purpose for this issue is proving the
curator contract holds for a second, structurally different consumer
(files-as-skills vs. observations-as-memory), not solving "what makes a
skill good."

internal/infrastructure/skillcurator/store.go: a curator.SkillStore
implementation over one root's .agents/skills/*/SKILL.md files. Not
internal/skill.Discover directly -- Discover parses every skill up front
and fails the WHOLE call if any one of them doesn't parse (correct for
"load the catalog to boot the daemon", wrong for "review each skill and
report what's wrong with it individually"). This store lists/reads raw
bytes per skill; the curator itself parses each one so one bad skill's
parse failure becomes one Action, not an aborted pass.

internal/infrastructure/skillcurator/skillcurator.go: the CuratorEngine.
Check is a cheap directory-listing bool. Pass classifies each skill into
exactly one of: skill.invalid (frontmatter parse failure, left
untouched), skill.incomplete (missing required name/description,
untouched), skill.normalized (trailing whitespace / missing final
newline, written back), or nothing (already clean). Manifest declares
only Skills -- no Tools, no MemoryEngine -- since nothing here needs
either.

Wired into internal/app/archied/bootstrap.go:setupCurators, rooted the
same way loadWorkflows already resolves skillsBase (cfg.SkillsDir falling
back to cfg.WorkDir) -- one local root, not the full multi-root catalog a
chat turn reads. Skills is now a shared registrar-level service
(alongside Events/MemoryEngines), narrowed out of a curator's view by
registry.filter when undeclared, same as every other capability.

task check is green (full Go suite + dashboard tests); go test ./...
is green.
2026-09-02 02:58:36 +10:00
sam 7b39f7bd0c feat(memory): builtin engine implementing the domain/memory contract
Part of archie-core-1786637499480 (ship the file-writing store as the
default memory engine).

Records the missing design decision first (migration-decisions.md#5,
"authoritative memory record and revision model" was explicitly open):
internal/memory/builtin.Store addresses blocks by content substring with
no stable per-block ID, which the new Record{ID}/Forget(id) contract
needs. Decided and implemented: an ID assigned once at Write via a
leading `<!--mem:<uuid>-->` marker line, stripped back out of
Record.Content on read, located by that marker (not content) for
Forget -- so an unrelated edit can never orphan a Forget. Identity maps
to a directory named hex(sha256(identity)), not the identity string, so
an arbitrary identity can never traverse outside the configured root.
Kind maps to a builtin section name via the store's own
ValidateSectionName, so an invalid Kind fails the same way an invalid
section already does.

internal/infrastructure/memory.BuiltinEngine wraps one builtin.Store per
identity and implements domain/memory.MemoryEngine end to end
(Write/Query/List/Forget, Lifecycle, Manifest). Lives under
internal/infrastructure per organisation.md ("infrastructure implements
domain contracts"), not internal/domain/memory, which stays
contract-only.

internal/memory/builtin is untouched -- confirmed via git status and its
own test suite passing unmodified, per this issue's acceptance
criterion. This engine is not yet wired into daemon bootstrap or the
config surface (archie-core-1786637490043); it proves the contract
holds for the builtin store the same way curator's reference
implementations prove the curator contract, and registers/runs cleanly
through domain/memory.Registry (TestBuiltinEngineRegistersAndRuns
ThroughTheRegistry).
2026-09-02 02:36:10 +10:00
sam 932a8b0680 docs(architecture): reconcile domain package map, add domain/messaging types
- migration-decisions.md: confirm curator/mapping/scheduling as
  legitimate domains that were never added to the confirmed list;
  document identity/agent/messaging/plugin as approved-but-unmaterialised
  targets rather than creating dead stub packages.
- Resolve the open Session/Messaging decision (section 2): keep
  internal/gateway as-is rather than risk a big-bang rewrite of live
  session/compression/approval/branch logic; materialise
  internal/domain/messaging with the already-settled canonical
  Message/Conversation/ToolCall/ToolResult types. Actual gateway
  migration onto these types is tracked as sequenced follow-up bd
  issues (label messaging-migration).
- ARCHITECTURE.md: add internal/gateway, internal/tools, internal/memory,
  internal/secret, internal/plugin, internal/skill to the Package Map.
2026-08-31 21:26:06 +10:00
sam 5ce821b561 chore: update issue tracker, remove ai-sdk ADR 2026-08-28 22:28:49 +10:00
sam 3b8ac3e8d1 docs: record ai-sdk to goai migration decision 2026-08-28 22:28:26 +10:00
sam 61e7fd3068 feat(webui): dashboard-aware chat — Archie sees the page, points you there
The web chat is now a persistent slide-over drawer available on any page
(not just the /chat route), and the agent is told which dashboard page the
operator is on so it can point them to the relevant page with a clickable chip.

Backend (gateway):
- Message gains a per-message Page field; the system prompt renders it in
  the <env> block (Current page), so Archie knows where the operator is.
- New dashboard_tools.go: a DashboardPages registry plus two web-only tools
  — page_index (lists every page and what it shows) and dashboard_navigate
  (validates a path, returns {path,label}; rejects unknown/empty paths).
- The web chat sink emits a dedicated navigate stream event when the model
  calls dashboard_navigate, independent of ShowToolCalls.

Frontend (webui):
- chat sends currentPage() (location.hash) with each stream request; a
  dashboard_navigate result renders as a clickable link chip (href=#/path).
- The chat is mounted once in a slide-over drawer with a topbar launcher,
  scrim and Esc-to-close; /chat opens the drawer instead of a second page.
- ui/dist rebuilt (committed) with the bundled changes.
- .gitignore: ignore .impeccable/ (impeccable tool critique snapshots).

New capability (per scope discipline): designed in
docs/prds/dashboard-aware-chat-navigation.md.

Tests: dashboard_tools_test (registry, resolve/reject, web-only gating),
prompt page-line cases, webui Message.Page + navigate-frame capture, and
chat-tools chip tests. task check is green.
2026-08-28 22:28:10 +10:00
sam 5e240721e2 feat(logging): cursor pagination and filtering for task_logs
task_logs could only read the newest N entries with no way to walk the
rest, so a task's log beyond the page cap was unreachable and the agent
had no honest way to say what was missing. Add logging.Page -- a
byte-offset cursor over the tail-most 8 MiB scan window -- plus
Query.Since/Until time bounds, and wire the gateway's task_logs tool to
expose level/since/until/after_id with cursor and more_available in the
result.

The reader now reads the scan window into a single in-memory buffer and
walks it by exact slice offsets. Offsets into a contiguous buffer are
trivially correct, which an incremental reader over the file is not:
bufio.Reader buffers ahead, so its underlying file position runs one
buffer past its logical position and cannot be used as a cursor. The
same rewrite removes f.Read in favour of f.ReadAt (full-or-error), since
a silent short read would drop the newest entries and still report
success.

The scan window is a documented hard ceiling: for a log over 8 MiB the
oldest bytes are unreachable by any cursor -- that is what Truncated
reports, and send_file remains the path for a complete archive. The
tool description teaches the model the difference between more_available
(more this call did not return) and truncated (older entries the scan
cap hides).

Closes archie-core-vq92 (#625).
2026-08-26 02:02:36 +10:00
sam a825db283c fix: harden task and chat execution
Task runs now preserve real model context limits, provider usage, baseline-repair spend, commit identity, durable agent-finish telemetry, and useful timeline details without imposing a token cap. Dashboard forge links are canonical per owning GitHub or Gitea identity. Chat turn claims acquire SQLite writer intent before reading, eliminating the concurrent SQLITE_BUSY snapshot race.
2026-08-25 05:48:17 +10:00
sam 64be0e7ec0 docs(prds): decide reviewer isolation via agent.Subagent, fail-closed
Section 1 of the adversarial self-review decision was superseded on
2026-08-24: it assumed daemon-owned in-process execution and a daemon-side
second Pool.Acquire, both removed by archie-core-q15y. It is now re-decided
rather than merely marked dead, so the document is Active again.

The reviewer is an agent.Subagent run from inside the task-scoped worker.
ai-sdk v0.1.22 is already pinned, so this needs no SDK bump, and it adds no
host executor, no second container acquire and no nested container lifecycle
-- the open question the supersession left.

Subagent.Run passes Prompt and never populates Messages, so conversation
isolation is a construction fact rather than prompt text. It does NOT give
filesystem isolation: the sub-agent shares the parent's process, working
directory and credentials, so the .git-stripped snapshot survives from the
original design and is load-bearing, and the reviewer's toolset must be
read-only, rooted at the snapshot, and must exclude the worktreerpc
publication grant -- otherwise a reviewer could push the branch it exists to
block. Run() not Tool(), so the implementer cannot elect to review itself.
Findings return through a closure-captured tool because Run yields only text,
and are read even when Run errors, since captured findings survive.

Also decides fail-closed: a review that fails to RUN parks the task rather
than opening an unreviewed PR. ReviewEnabled is opt-in, so an operator who
turned the gate on wants the gate. This is why h019.2 models "ran and found
nothing" separately from "did not run" -- collapsing them would make an
outage indistinguishable from a clean review.

h019.3 is unblocked and rewritten with the construction rules. h019.4's
description said the task parks as waiting_human, contradicting section 4's
decision of StatusParked; corrected, and the fail-closed rule added.
2026-08-25 00:56:10 +10:00
sam 972fb5b8e1 feat(deployment): default to managed NATS workers
Autonomous workflows now always hand a complete task to a scoped archie-agent container. Embedded NATS is the default and external NATS changes only broker placement; legacy agent and containers.enabled inputs decode for migration but cannot select an execution path. The daemon flushes task-event subscriptions before a worker can publish, and runtime updates snapshot and restore the configured worker image.
2026-08-24 14:56:57 +10:00
sam 5d00ec7b0c refactor(agentexec): remove legacy execution paths
Autonomous workflows now have exactly one production boundary: a task-scoped archie-agent receives a full-task core-NATS request and runs the worker-local stage loop. The legacy [agent] section remains decode-only for migration; it no longer selects runtime behavior.
2026-08-24 14:56:57 +10:00
sam 1b7bc004b6 feat(nats): expose embedded broker to managed workers 2026-08-24 14:56:57 +10:00
sam 691e10c251 fix(telegram): compact tool output and drop turn caps
Render completed ToolCallEvents as Telegram-safe streaming blocks
with compact name/status and a short cleaned preview. Summarize
file contents and listings instead of dumping source or quoted
JSON. Collapse repeated legacy turn-budget failures.

Remove max_tokens turn gating and the 200000 turn_budget_chars
default so absent or commented config cannot restore an aggregate
cap. Keep per-result max_result_chars and spill.

Confine find/grep discovery to the active workspace/worktree.

Refs #604
2026-08-23 16:40:16 +00:00
sam c7b2c2bf46 feat(forge): observable webhook receiver + dedup regression proof; decide adversarial review design
archie-core-7d5u.5 (idempotency and poll/webhook parity): the dedup
mechanism (TaskEnvelope.IdempotencyKey + PublishUnique + JetStream
Nats-Msg-Id dedup) and the shared dispatch predicate were already fully
wired by 7d5u.4. What was missing: proof the dedup actually holds, and
observability for a wedged receiver.

- internal/infrastructure/eventbus/nats/dedup_test.go: regression test
  proving PublishUnique suppresses a duplicate publish on the same
  WorkQueuePolicy stream ARCHIE_TASKS uses in production.
- internal/forge/webhook/receiver.go: Receiver.Status() tracks
  authenticated deliveries separately from successful publishes; a GET
  to the receiver's listen address (GitHub only ever POSTs) now returns
  it as JSON, so a wedged/non-delivering receiver is observable instead
  of indistinguishable from an idle repo.

Poll/webhook parity (disabling webhook still polls) needed no code
change: Cycle()/Run() never reference cfg.Forge.Intake anywhere in
internal/daemon, only setupForgeWebhook gates the webhook HTTP server.

archie-core-h019.1 (design: adversarial self-review stage): one-page
decision at docs/prds/adversarial-self-review.md. Reviewer runs in a
second workspace pointed at a .git-stripped snapshot of the reviewed
commit, never the implementer's worktree, so isolation is a property of
what's mounted rather than a prompt instruction. Findings contract is
workflow.ReviewFinding, owned by internal/domain/workflow. A surviving
error-level finding sets StatusParked, not WaitingHuman -- the epic's
informal wording named two different states in the actual machine; this
resolves which.

Both changes reviewed by a fresh subagent with no memory of the
implementation, per CLAUDE.md's adversarial-verification requirement;
zero blocking findings, two minor coverage gaps closed.
2026-08-22 19:45:31 +10:00
sam aba035a396 feat(webui): payload field mapping -- bind JSON paths to named fields, preview before saving
Adds t2db.3: schema-by-example field mapping for the no-code playbook epic.
An operator picks a captured event, clicks through its payload to bind named
fields to JSON paths, and previews the resolution against a real event
before saving -- a missing, null, or type-changed path fails loudly (named
by field and event) rather than handing an agent a blank value.

- internal/domain/mapping: Mapping/Field types, path parsing/walking,
  Resolve (per-field failure reporting), Validate (incl. duplicate names)
- internal/store/mappings.go: SQLite persistence, mirrors captures.go
- internal/webui/api_mapping.go: CRUD + preview HTTP API
- ui/src/mappings/: dashboard editor (clickable payload tree, live preview)

Design decisions recorded in docs/prds/payload-field-mapping.md. t2db.4
(binding a mapping + matcher to a workflow) is out of scope here.
2026-08-22 19:18:15 +10:00
sam c7987f0595 feat(nats): run an embedded server when no external NATS is configured
Wire github.com/nats-io/nats-server/v2 into production startup so a
single-process deployment gets reaction delivery and task distribution
over an in-process server rather than a separate one
(docs/prds/embedded-nats.md).

Meaning change: [nats].url empty used to mean "no NATS, use the SQLite
ClaimNext flow"; it now means "start an embedded NATS server". The old
behaviour is the explicit opt-out [nats].mode = "off". mode resolves from
url when unset (url set -> external, url empty -> embedded), and
agent.mode="nats" / [containers] now require mode="external" (an embedded
loopback server cannot be reached by agent workers or containers).

The daemon owns the embedded server's lifecycle: StartEmbedded before
dialing the client, Shutdown after the client closes (LIFO cleanups). The
embedded server binds loopback on a random port, enables JetStream in a
store dir under the data directory, and leaves its logger nil so a startup
failure surfaces as a Connect error rather than an os.Exit.

Retention becomes a nats.Config field (default WorkQueuePolicy) so a future
reaction stream can fan out under LimitsPolicy instead of the hardcoded
competing-consumer policy that starves a second consumer on overlapping
subjects.
2026-08-22 05:18:31 +10:00
sam 37bf8ef05a feat(capture): store unbound webhook events, retention-bounded
Adds POST /webhooks/capture/{source}: an unauthenticated endpoint (mounted
alongside /healthz, bypassing dashboard token auth by design) that persists
any inbound webhook archie has no binding for yet, so it can be inspected
and mapped later (t2db.2/t2db.3).

docs/prds/event-capture-storage.md decides SQLite over NATS JetStream for
storage: NATS is not unconditionally available until 7d5u.2 ships embedded
support, and capture's single-writer/single-reader access pattern doesn't
need JetStream's fan-out/replay machinery anyway.

internal/store/captures.go adds a captured_events table with prune-on-write
(retention age + row-count cap, both configurable). The HTTP handler applies
webhookguard redaction before persisting, and enforces two more independent
disk-bound mechanics before that: a per-request body-size cap and a
per-remote-address rate limit. New [capture] config section with documented
defaults; all fields optional, capture is on by default.

Adversarial review (manual, after the usual fresh-subagent pass hit a
session API limit mid-run) caught and fixed three real defects before this
landed, each with a regression test confirmed to fail against the pre-fix
code:
- the body-size cap silently no-op'd when CaptureMaxBodyBytes was left
  unset, the one bound that limits a single payload's size rather than row
  count or write rate
- the rate limiter was keyed by the request's own source path segment --
  unregistered and attacker-chosen by design for this endpoint, violating
  webhookguard.RateLimiter's documented bounded-key-space assumption and
  letting a sender bypass the limit by rotating the segment; rekeyed to
  remote address
- the retention prune compared time.RFC3339Nano strings, whose trimmed
  variable-width fractional seconds break lexicographic ordering within the
  same second; switched to fixed-width RFC3339

Closes archie-core-t2db.1.
2026-08-22 04:51:25 +10:00
sam c1ea3db532 docs(prds): decide event sources and reaction delivery
archie-core-7d5u.1 -- closes with the decision recorded. Investigated three
mechanisms already in the tree rather than designing from scratch:

- No out-of-process extension boundary exists or is needed; in-process typed
  families throughout, and plugins-and-extensions.md does not need amending.
- Reaction delivery reuses internal/eventbus's Publisher/Consumer contract,
  but NOT the existing nats.Client unmodified: it hardcodes
  jetstream.WorkQueuePolicy (client.go:65), the competing-consumer policy
  correct for ARCHIE_TASKS and wrong for fan-out reactions -- two reactions on
  overlapping subjects would hit the exact silent-starvation trap CLAUDE.md
  already documents. Retention needs to become a Config field; reactions get
  a second stream.
- Reactions in these two epics are producer-only (curator's WakeOnPrimaryInput
  for cheap wake, or the existing pollNATS->publishTask shape for real work).
  No veto/mutate-in-flight capability is being built here.
- No single generic Source interface. workintake.TaskEnvelope already is
  forge intake's typed contract and needs no new type -- its IdempotencyKey is
  keyed on owner/repo/number, not delivery source, so a webhook-sourced
  envelope dedups against a poll-discovered one for free. What's genuinely
  shared is a small cross-cutting package for HMAC/rate-limit/capture/approval
  mechanics, owned by neither domain.

Retitled and rescoped archie-core-7d5u.3 to match: it was still asking for
'one registry for every kind of inbound caller,' which this decision
explicitly rejects. Removed a false blocking dependency of t2db.1 on 7d5u.4
(GitHub-specific receiver) -- they are siblings sharing the new cross-cutting
package, not a chain -- and linked them instead.
2026-08-22 01:01:52 +10:00
sam efb8ae7a8d docs(prds): decide webhook intake security ahead of the code it constrains
archie-core-t2db.5 -- decides the five non-negotiables its description named
before capture (t2db.1) or the intake family (7d5u.3) exist to build against
it: per-source HMAC auth reusing internal/channels/webhook's existing scheme,
a draft/pending_approval/armed binding state machine modeled on telegram's
dangerous-command approval flow, per-source rate limiting (precedent:
pairing.MaxPendingPerPlatform) that rejects rather than silently drops,
blast-radius bounding via the same gate/diff-cap/sandbox every task already
gets rather than a new relaxation, and 7-day retention with best-effort
redaction reusing gateway/stream.go's sensitiveParameterKey heuristic.

The bead stays open: its code-level acceptance criteria constrain beads that
haven't landed yet and cannot be satisfied by a document alone.
2026-08-22 00:54:00 +10:00
sam 45345cdfb6 refactor(agent): move worker runtime into application layer
cmd/archie-agent used to own worker construction, NATS lifecycle, task execution, MCP setup, and workflow wiring. It now owns process input only and invokes internal/app/agentworker, while transport-specific NATS, boot-brief, and git-safety mechanics live under infrastructure.
2026-08-15 00:43:12 +10:00
sam 99fb1102cc docs(architecture): mark the config-overlay subset implemented on safe-change-and-recovery 2026-08-10 01:46:18 +10:00
sam 5343c68160 docs(architecture): record the implemented config reload and runtime overlay
Per the sequencing call (adversarial pass first, docs last), the docs
now reflect the design the reviews confirmed.

configuration.md:
- Mutation ledger gains rows for SIGHUP reload, dashboard PATCH,
  dashboard reset, and the runtime overlay store, each mapping to the
  universal change protocol with what is and is not yet implemented.
- New section 'Implemented runtime reload and config overlay (2026-08)':
  the Holder published-snapshot model, the reloadable-field criterion
  (a field is reloadable when EVERY consumer re-reads it), SIGHUP
  reload semantics, the overlay store (separate -config.sqlite, dotted
  keys, denylist, bootstrap degrade), and the PATCH/reset paths.

safe-change-and-recovery.md:
- New section 'Runtime config overlay: degrade paths and recovery'
  tabulating the three boot degrade paths and the reload degrade, all
  verified by the adversarial passes, plus the three documented
  recoveries (--no-config-overlay flag, rm the overlay file, dashboard
  reset). Status line notes the overlay subset is implemented while
  the full runtime-supervision protocol remains under design.

No new documents, per the constraint.
2026-08-10 01:45:44 +10:00
sam 61b4b7b0cf refactor(store): replace legacy persistence with SQLite
Task and event persistence now uses a dedicated SQLite sidecar derived from the configured state prefix, matching the independently persisted conversation store. The configured path now means a state-path prefix rather than a database file so fresh SQLite state never attempts to interpret the previous storage format.

Remove the retired implementation and dependency completely, strengthen SQLite migrations and concurrency behavior, restore backend-neutral coverage, and align configuration, operational guidance, UI labels, and deployment documentation with the new storage and host-daemon topology.
2026-08-09 22:27:25 +10:00
sam 95c5d59e7c docs(architecture): mark observability design as implemented 2026-08-09 19:29:33 +10:00
sam fab8a66486 Merge pull request #457 from samcharles93/fix/452-task-log-sink
feat(logging): add a per-task log sink
2026-08-09 18:17:24 +10:00
sam fd4420a038 docs(architecture): record decisions recovered from the pre-migration tracker
Eight approved decisions existed only as beads memories and were lost when
the tracker was removed. Recovered from a backup taken six minutes before
8bf0f6b and folded into the docs that own them, rather than a new document.

- migration-decisions: recorded position as of 2026-07-29, the deferred
  private-input-DTO compromise, and the agreed internal/app/{archied,
  agentworker} next step
- configuration: missing credential degrades a capability, only invalid
  config is fatal; config generation moves from install.sh to archied setup
- identity: the three-layer invariants/SOUL/representation model, and why
  a curator may only propose SOUL changes
- agent-system: the curator isolation invariant (curators are peers, never
  on a chat turn's critical path)
- messaging-and-work-intake: semantic search over messages is an explicit
  non-goal, with the measurements behind it
- safe-change-and-recovery: the /restart deadlock and lockout constraints
- plugins-and-extensions: dates the surprisal deferral

Also replaces the dead `**Beads issue:** archie-core-5d7` frontmatter across
all 12 documents with the GitHub issue it migrated to (#73), and rewrites
every other beads identifier to its issue number.
2026-08-09 18:10:12 +10:00
sam 86d3071f39 fix(workflow): carry gate output into the baseline park error
StageBaselineGate captured a failing gate command's full output but
discarded it: a 200-char clip went into a Warn log, and the terminal
error carried only res.Status. A park reason like "go build ./...
fails ... (status: failed)" gave no way to see the actual compiler
error, and the container that had it is already gone (AutoRemove).

Now the park error carries a bounded tail of the real output (tail,
not head, since a failing test's explanation is at the end after
pages of "ok" lines), rune-safe so it can't emit invalid UTF-8 into
a persisted park reason. The three existing byte bounds in this
stage (log line, agent mission, park error) are named and documented
as independently sized rather than left as bare numbers.

Also adds docs/architecture/observability.md, the design doc for the
larger task-observability epic (#450) this fix is the first step of.
2026-08-09 16:23:24 +10:00
sam 308666b410 chore: remove legacy terminology and dead code
- Reword docs and comments to drop vague "legacy" qualifiers where they no longer describe deprecation status
- Remove unused code: RecentEvents, MessageEvent.ToLegacy, index setState, stderr redaction tool, telegramOnlyCommands
- Delete or comment out tests covering removed or superseded behavior
2026-08-06 22:55:26 +10:00
sam 0ceab9d897 fix(nell): move ID counters off SDK-reserved meta: prefix (archie-core-7ma)
NellDB v0.3.0 skips meta:* ids when rebuilding its rev cache on open, so
after any restart the counter Get returned a _rev with no cache entry and
every RMW Put failed with sdk: revision conflict -- task enqueues and
event inserts were permanently broken.

- Counters move to counter:task_id / counter:event_id.
- migrateCounters seeds the new keys from the legacy docs at open; any
  read/write/validation failure aborts OpenStore so allocation can never
  silently restart at 1. Legacy docs stay in place, skipped by scans.
- isReservedKey matches the counter ids exactly (a counter: prefix rule
  would hide every task in a repo owned by "counter").
- Existing counters are validated (next_id >= 1) at open and at allocation.
- EnqueueIssue persists identity, which it previously dropped.
- Tests: reopen regression with exact post-reopen IDs and event
  preservation, migration sequence carry-over for both counters,
  fail-closed on read/write/invalid values, counter-owned repos,
  identity round-trip.

Adversarial review: zero surviving findings.
2026-08-05 19:55:06 +10:00
sam 4ece064026 feat(curator): engine family contract, registry, and runtime (archie-core-rfp, archie-core-89x)
internal/curator is a conforming engine family per the strict plugin
engine rule: typed CuratorEngine contract (Lifecycle + domain ops),
owning Registry (registration, discovery, start/health/stop with
failure isolation, reverse-order shutdown, ctx-bounded stop), and a
narrow typed Registrar — each curator receives a capability-filtered
view of the host at Bind, never the daemon.

Runtime: one loop per curator on an injectable clock (fake-clock
testable), best-effort wake nudges with the trigger decision kept in
Check (a dropped nudge only delays to the next check-in, never
corrupts accounting), per-pass timeout and a max-concurrent-passes
slot so curator bursts cannot starve chat turns, per-pass panic
recovery, and bounded shutdown even when a pass ignores cancellation.
curator_run / curator_action / curator_error ride the in-process
event bus so activity stays attributable.

Wired in cmd/archied/main.go (constructed beside memory, Start after
daemon startup, shutdown ordered before memory/LLM teardown) and
exposed as daemon.Daemon.Curators.
2026-08-05 14:54:12 +10:00
sam 2307c8c81a feat: unify provider and model selection 2026-07-31 03:40:03 +10:00
sam 8ea6367ee0 chore: remove stray Tau CLI manual from archie-core docs
docs/ is the archie-core VitePress site (Archie-branded, see index.md).
This README.md was a Tau reference manual that was never linked from the
site index and doesn't belong in this repo.

The tau references in internal/tools/builtin/ are legitimate attribution
headers — those files were deliberately lifted from tau with source
commit hashes.
2026-07-31 02:21:49 +10:00
sam 6be754787d polish: add install.sh, XDG setup, deployments/ examples, and github-token docs\n\n- Add install.sh following XDG Base Directory specification (~/.config/archie, ~/.local/share/archie, ~/.local/bin)\n- Add interactive setup for code forge (GitHub / Gitea / Standalone) and LLM provider selection (Ollama model picker / Cloud keys)\n- Implement NoopForge in internal/forge/noop.go for running Archie in standalone mode without a forge\n- Enable loginctl user linger and systemd user unit setup in install.sh\n- Add docs/github-token.md with step-by-step guidance for Classic PAT generation\n- Add top-level deployments/ directory with scenario templates (single-forge GitHub, multi-forge GitHub+Gitea, local Ollama standalone, systemd runbook) 2026-07-30 01:25:46 +10:00