Commit Graph
41 Commits
Author SHA1 Message Date
sam d28ca099c5 refactor(agent): move worker runtime into application layer
cmd/archie-agent used to own worker construction, NATS lifecycle, task execution, MCP setup, and workflow wiring. It now owns process input only and invokes internal/app/agentworker, while transport-specific NATS, boot-brief, and git-safety mechanics live under infrastructure.
2026-08-15 00:43:12 +10:00
sam 99fb1102cc docs(architecture): mark the config-overlay subset implemented on safe-change-and-recovery 2026-08-10 01:46:18 +10:00
sam 5343c68160 docs(architecture): record the implemented config reload and runtime overlay
Per the sequencing call (adversarial pass first, docs last), the docs
now reflect the design the reviews confirmed.

configuration.md:
- Mutation ledger gains rows for SIGHUP reload, dashboard PATCH,
  dashboard reset, and the runtime overlay store, each mapping to the
  universal change protocol with what is and is not yet implemented.
- New section 'Implemented runtime reload and config overlay (2026-08)':
  the Holder published-snapshot model, the reloadable-field criterion
  (a field is reloadable when EVERY consumer re-reads it), SIGHUP
  reload semantics, the overlay store (separate -config.sqlite, dotted
  keys, denylist, bootstrap degrade), and the PATCH/reset paths.

safe-change-and-recovery.md:
- New section 'Runtime config overlay: degrade paths and recovery'
  tabulating the three boot degrade paths and the reload degrade, all
  verified by the adversarial passes, plus the three documented
  recoveries (--no-config-overlay flag, rm the overlay file, dashboard
  reset). Status line notes the overlay subset is implemented while
  the full runtime-supervision protocol remains under design.

No new documents, per the constraint.
2026-08-10 01:45:44 +10:00
sam 61b4b7b0cf refactor(store): replace legacy persistence with SQLite
Task and event persistence now uses a dedicated SQLite sidecar derived from the configured state prefix, matching the independently persisted conversation store. The configured path now means a state-path prefix rather than a database file so fresh SQLite state never attempts to interpret the previous storage format.

Remove the retired implementation and dependency completely, strengthen SQLite migrations and concurrency behavior, restore backend-neutral coverage, and align configuration, operational guidance, UI labels, and deployment documentation with the new storage and host-daemon topology.
2026-08-09 22:27:25 +10:00
sam 95c5d59e7c docs(architecture): mark observability design as implemented 2026-08-09 19:29:33 +10:00
sam fab8a66486 Merge pull request #457 from samcharles93/fix/452-task-log-sink
feat(logging): add a per-task log sink
2026-08-09 18:17:24 +10:00
sam fd4420a038 docs(architecture): record decisions recovered from the pre-migration tracker
Eight approved decisions existed only as beads memories and were lost when
the tracker was removed. Recovered from a backup taken six minutes before
8bf0f6b and folded into the docs that own them, rather than a new document.

- migration-decisions: recorded position as of 2026-07-29, the deferred
  private-input-DTO compromise, and the agreed internal/app/{archied,
  agentworker} next step
- configuration: missing credential degrades a capability, only invalid
  config is fatal; config generation moves from install.sh to archied setup
- identity: the three-layer invariants/SOUL/representation model, and why
  a curator may only propose SOUL changes
- agent-system: the curator isolation invariant (curators are peers, never
  on a chat turn's critical path)
- messaging-and-work-intake: semantic search over messages is an explicit
  non-goal, with the measurements behind it
- safe-change-and-recovery: the /restart deadlock and lockout constraints
- plugins-and-extensions: dates the surprisal deferral

Also replaces the dead `**Beads issue:** archie-core-5d7` frontmatter across
all 12 documents with the GitHub issue it migrated to (#73), and rewrites
every other beads identifier to its issue number.
2026-08-09 18:10:12 +10:00
sam 86d3071f39 fix(workflow): carry gate output into the baseline park error
StageBaselineGate captured a failing gate command's full output but
discarded it: a 200-char clip went into a Warn log, and the terminal
error carried only res.Status. A park reason like "go build ./...
fails ... (status: failed)" gave no way to see the actual compiler
error, and the container that had it is already gone (AutoRemove).

Now the park error carries a bounded tail of the real output (tail,
not head, since a failing test's explanation is at the end after
pages of "ok" lines), rune-safe so it can't emit invalid UTF-8 into
a persisted park reason. The three existing byte bounds in this
stage (log line, agent mission, park error) are named and documented
as independently sized rather than left as bare numbers.

Also adds docs/architecture/observability.md, the design doc for the
larger task-observability epic (#450) this fix is the first step of.
2026-08-09 16:23:24 +10:00
sam 308666b410 chore: remove legacy terminology and dead code
- Reword docs and comments to drop vague "legacy" qualifiers where they no longer describe deprecation status
- Remove unused code: RecentEvents, MessageEvent.ToLegacy, index setState, stderr redaction tool, telegramOnlyCommands
- Delete or comment out tests covering removed or superseded behavior
2026-08-06 22:55:26 +10:00
sam 0ceab9d897 fix(nell): move ID counters off SDK-reserved meta: prefix (archie-core-7ma)
NellDB v0.3.0 skips meta:* ids when rebuilding its rev cache on open, so
after any restart the counter Get returned a _rev with no cache entry and
every RMW Put failed with sdk: revision conflict -- task enqueues and
event inserts were permanently broken.

- Counters move to counter:task_id / counter:event_id.
- migrateCounters seeds the new keys from the legacy docs at open; any
  read/write/validation failure aborts OpenStore so allocation can never
  silently restart at 1. Legacy docs stay in place, skipped by scans.
- isReservedKey matches the counter ids exactly (a counter: prefix rule
  would hide every task in a repo owned by "counter").
- Existing counters are validated (next_id >= 1) at open and at allocation.
- EnqueueIssue persists identity, which it previously dropped.
- Tests: reopen regression with exact post-reopen IDs and event
  preservation, migration sequence carry-over for both counters,
  fail-closed on read/write/invalid values, counter-owned repos,
  identity round-trip.

Adversarial review: zero surviving findings.
2026-08-05 19:55:06 +10:00
sam 4ece064026 feat(curator): engine family contract, registry, and runtime (archie-core-rfp, archie-core-89x)
internal/curator is a conforming engine family per the strict plugin
engine rule: typed CuratorEngine contract (Lifecycle + domain ops),
owning Registry (registration, discovery, start/health/stop with
failure isolation, reverse-order shutdown, ctx-bounded stop), and a
narrow typed Registrar — each curator receives a capability-filtered
view of the host at Bind, never the daemon.

Runtime: one loop per curator on an injectable clock (fake-clock
testable), best-effort wake nudges with the trigger decision kept in
Check (a dropped nudge only delays to the next check-in, never
corrupts accounting), per-pass timeout and a max-concurrent-passes
slot so curator bursts cannot starve chat turns, per-pass panic
recovery, and bounded shutdown even when a pass ignores cancellation.
curator_run / curator_action / curator_error ride the in-process
event bus so activity stays attributable.

Wired in cmd/archied/main.go (constructed beside memory, Start after
daemon startup, shutdown ordered before memory/LLM teardown) and
exposed as daemon.Daemon.Curators.
2026-08-05 14:54:12 +10:00
sam 2307c8c81a feat: unify provider and model selection 2026-07-31 03:40:03 +10:00
sam 8ea6367ee0 chore: remove stray Tau CLI manual from archie-core docs
docs/ is the archie-core VitePress site (Archie-branded, see index.md).
This README.md was a Tau reference manual that was never linked from the
site index and doesn't belong in this repo.

The tau references in internal/tools/builtin/ are legitimate attribution
headers — those files were deliberately lifted from tau with source
commit hashes.
2026-07-31 02:21:49 +10:00
sam 6be754787d polish: add install.sh, XDG setup, deployments/ examples, and github-token docs\n\n- Add install.sh following XDG Base Directory specification (~/.config/archie, ~/.local/share/archie, ~/.local/bin)\n- Add interactive setup for code forge (GitHub / Gitea / Standalone) and LLM provider selection (Ollama model picker / Cloud keys)\n- Implement NoopForge in internal/forge/noop.go for running Archie in standalone mode without a forge\n- Enable loginctl user linger and systemd user unit setup in install.sh\n- Add docs/github-token.md with step-by-step guidance for Classic PAT generation\n- Add top-level deployments/ directory with scenario templates (single-forge GitHub, multi-forge GitHub+Gitea, local Ollama standalone, systemd runbook) 2026-07-30 01:25:46 +10:00
sam 69640b5216 docs: consolidate into a single docs/ tree
Fold the old docs/ into the VitePress site and rename docs-2 -> docs.
The 12 architecture PRDs become first-class pages at docs/architecture/
with sidebar nav, rather than being buried under an archive path.
Historical material stays at docs/archive/. Rewrites every docs-2 and
docs/prds/architecture reference across the repo. Site builds clean.
2026-07-29 03:17:42 +10:00
sam d1bbd37d95 chore: cleanup 2026-07-29 02:10:44 +10:00
sam 65aa5b462d chore: cleanup 2026-07-28 18:46:50 +10:00
sam 647537c00a refactor(telegram): keep one model selector command 2026-07-27 11:44:27 +10:00
sam f73b2d1c8c feat: memory manager sync/prefetch/scan/shutdown + broad in-flight epic work
Checkpoint commit covering concurrent work across several archie-core-abg
epics to avoid losing in-progress state:

- Memory manager (18.7-18.16): background sync pipeline, prefetch with
  timeout, memory-write notification, threat scanning, shutdown drain
  with abandoned-write tracking, frozen snapshot pattern. Wired
  ScanContent and NotifyMemoryToolWrite into HandleToolCall so scanning
  and notification actually apply to writes, not just exist as unused
  primitives.
- Fixed a literal unresolved merge-conflict that had been committed to
  internal/memory/scrubber/scrubber.go.
- Tool provider/registry, MCP client/transport (stdio, HTTP, SSE),
  multimodal results, disclosure.
- Pairing device auth store, rate limiting.
- Gateway compression, session store, tasks.
- Various workflow, plugin host, and agentexec updates.
2026-07-27 03:34:46 +10:00
sam f6d51416f0 chore: replace 607 em-dashes with double hyphens project-wide
Includes MCP crash-after-init test fix (crash moved from handle() to serve()
entry point so transport sees immediate subprocess death).
2026-07-26 09:53:33 +10:00
sam 5396436d35 docs: Hermes feature gap analysis — 60 gaps across 11 subsystems
7 subagents mapped the full Hermes codebase (~200 .py files, ~500K LOC).
Master gap analysis identifies 60 missing features across:
  P0 (7): adapter lifecycle, message model, sessions, platform config,
         tool registry, MCP client, memory provider ABC
  P1 (13): send result, streaming, dedup, media, guardrails,
           progressive disclosure, session history, pairing/auth,
           CLI, config subcommands, profiles, readiness
  P2 (22): interactive UX, subagents, browser/search/file tools,
           compression, cron, dashboard, API server, delivery
  P3 (3): billing, OAuth, desktop app

15 new beads issues created under the abg epic. NellDB identified
as the persistent memory/config backend (P0 integration issue).
2026-07-26 00:03:42 +10:00
sam e4c008af5e docs: multi-agent collaboration PRD from discovery interview
Discovery interview (archie-core-abg.1) confirmed scope: daemon-side
concurrency, cross-repo/config flexibility, agent-to-agent handoff,
chat channels with full pipeline parity, and general extensibility.
Design answer confirms per-identity goroutine with shared
infrastructure for failure isolation.
2026-07-25 23:50:45 +10:00
sam e4368f70af feat: persistent volumes with TTL for container/worktree storage (archie-core-j7v)
Add [containers].volume_ttl (default 72h). Persistent-storage repos get a
per-repo bare Git cache (--reference-if-able --dissociate) so task
worktrees clone fast without repeated full network clones, while staying
fully independent of the cache's lifecycle — cache expiry can never
corrupt an active or parked task's worktree. Docker repo volumes are
labeled as Archie-owned and swept non-forced (a volume still attached to a
running container survives and retries next cycle); the repo cache
directory is swept by mtime. Both cleanups run at daemon startup and each
poll cycle.

Filed archie-core-756 (repo-wide golangci-lint baseline) and archie-core-340
(intermittent JetStream TempDir teardown flake) as separate follow-ups
rather than scope-creeping them into this change.
2026-07-24 10:22:11 +10:00
sam 4be28e0e48 feat: optional NATS token authentication (archie-core-l1q)
Add token_env to [nats] config, resolved and validated at daemon startup
(fails early if configured but the named env var is unset). Container
workers get NATS_URL/NATS_TOKEN; archie-agent honors both while CLI-supplied
URL still takes precedence. Anonymous auth remains the default for
single-node deployments.

Unblocks archie-core-5zh (promote archie-agent to per-task process).
2026-07-24 07:51:06 +10:00
sam b6b23bbb59 feat: per-repo allow_concurrent override for same-repo dispatch (archie-core-g1l)
Add config.Repo.AllowConcurrent (allow_concurrent toml key). Repos opted in
skip the daemon's default FIFO same-repo task serialization while still
respecting [containers].max_concurrency globally. Worktrees are already keyed
per issue number, so opted-in repos don't collide on disk.
2026-07-24 07:47:10 +10:00
sam f8df051b19 feat: bounded concurrent task dispatch in daemon (archie-core-u64)
Daemon previously processed one task at a time. Add bounded concurrent
dispatch for SQLite and NATS agent modes, reusing [containers].max_concurrency
(0 = unlimited). Same-repository tasks stay FIFO-serialized to avoid worktree
conflicts; different repositories now run concurrently.

Unblocks archie-core-l1q (NATS transport) and archie-core-g1l
(per-repo allow_concurrent override).
2026-07-24 07:42:13 +10:00
sam d9923109e5 docs: record CreateIssue, Dolt, and Beads decisions 2026-07-23 19:12:38 +10:00
sam 16ca05da04 docs: add design decisions log, uncap token budget to 100M 2026-07-23 18:40:21 +10:00
sam f38ba5d515 feat: wire Layer 2 daemon plugin loader into archied startup
Daemon plugins (Yaegi-interpreted .go files) are now loaded from the
configured plugin_dir at startup. Each file must export a 'Plugin'
variable using the _Plugin wrapper struct, which bridges Yaegi types
to the Go plugin.Plugin interface.

Changes:
- internal/plugin/pluginextract/ — generated Yaegi symbol table for
  the plugin package (mirrors wfextract pattern), including the
  _Plugin interface wrapper that lets interpreted code implement
  Go interfaces
- LoadDir now accepts optional symbol tables via variadic parameter,
  avoiding an import cycle between plugin and pluginextract
- Plugin tests rewritten as behavioral (not structural), verifying
  actual Yaegi interpretation of valid, invalid, and missing plugins
- config.PluginDir field for specifying the plugin directory
- daemon.PluginRegistry field for holding loaded plugins
- main.go wires plugin loading: LoadDir at startup, register into
  daemon, log each loaded plugin

Why: the Plugin interface and LoadDir existed but weren't connected
to the daemon. Skills define workflows; daemon plugins are the
extension point for forge resolvers, storage backends, and other
core-daemon concerns.
2026-07-22 04:11:02 +10:00
sam 121dd7d815 docs: sync PRD status table and lifecycle section with post-de57f72 implementation 2026-07-22 02:45:48 +10:00
sam ac44d56965 docs: finalise v2 architecture PRD
Resolve all seven open questions with concrete answers:
- Agent image versioning: tagged versions, :latest dev-only
- NATS auth: anonymous single-node, credentials for multi-node
- Worktree ownership: answered by current impl (fresh clone per task)
- Grace period: reset on human reply, cap with max_total_uptime
- Concurrent agents: blocked by default, per-repo override
- Plugin architecture: extend existing Yaegi surfaces, don't replace
- Skill loading scope: parse frontmatter + inject into prompt for v2

Status: Draft -> Final.
2026-07-22 01:38:50 +10:00
sam 334cd78471 docs: replace all unicode characters with ASCII equivalents 2026-07-22 01:30:31 +10:00
sam adcdcb4b76 docs: resolve all seven open questions with concrete answers 2026-07-22 01:26:45 +10:00
sam c6372ef886 docs: reconcile v2 architecture PRD with actual implementation state
Adds an implementation-status section and annotates each section
(binary split, NATS, container sandbox, /data volume, plugin layers,
skills, task lifecycle, phases) with what's actually built today vs.
aspirational. Reconciles the plugin design with the sibling
yaegi-plugin-system.md PRD, whose three phases are already
implemented but scoped narrower than this doc's Layer 1/2 split.
2026-07-22 01:07:33 +10:00
sam c0d108a5d0 feat: add Yaegi-based plugin system (gates, custom stages, skill scripts)
Implements the full PRD (docs/prds/yaegi-plugin-system.md): repos can
now define .archie/gate.go for diff-inspecting quality rules shell
commands can't express, .archie/stages/*.go for custom pipeline steps,
and skills can ship Go scripts run via a new run_go_script agent tool —
all interpreted in-process with Yaegi, with panics recovered so a bad
script can't take down the daemon. Hot reload (phase 4) needed no code:
every loader reads from disk per call and every task gets a fresh
clone, so nothing is ever cached to go stale.
2026-07-22 00:54:46 +10:00
sam 5fc61e0bad feat: redesign architecture doc with interactive SVG 2026-07-21 23:44:48 +10:00
sam 1d995c587d docs: archie-core v2 end-state architecture diagram 2026-07-21 20:05:52 +10:00
sam 276f96fec3 docs: archie-core v2 architecture design doc (daemon/agent split, NATS, plugins, skills) 2026-07-21 19:48:38 +10:00
sam 3080adf87e docs: add TypeScript plugin engine PRD (goja + esbuild) 2026-07-21 17:49:50 +10:00
sam 1e73507d80 docs: add architecture docs, skills, and Yaegi PRD 2026-07-21 17:34:38 +10:00
sam 5cb4f32a8d docs: add PRD docs and move DESIGN.md to docs/ 2026-07-21 17:30:43 +10:00