Commit Graph
192 Commits
Author SHA1 Message Date
sam 52cc8f8a14 feat: memory P1 — sync pipeline, prefetch, notify, frozen snapshot, threat scanning, shutdown drain (18.7-18.9, 18.14-18.16)
sync.go — background serialized writer (single goroutine) for async
memory writes. Prefetch with 8s timeout per provider.

scanner.go — threat pattern scanning for memory content.

manager.go — notify_memory_tool_write bridging built-in writes to
external providers. Shutdown drain with 5s timeout.

All 6 impl + 6 verify beads closed. Memory epic complete.
2026-07-26 09:22:12 +10:00
sam 628b44387a beads: sync after memory scrubber + schema + builtin tool 2026-07-26 08:58:27 +10:00
sam 942deb9336 feat: built-in file-backed memory tool (18.12, 18.13)
internal/memory/builtin/provider.go — MemoryProvider backed by
MEMORY.md + USER.md files. LLM-callable memory tool with
add/replace/remove actions using section-sign delimited entries
and substring matching.

internal/memory/builtin/store.go — file read/write with atomic
replace and chmod 0600.
2026-07-26 08:58:27 +10:00
sam e8fe1b12a7 feat: tool schema normalization for memory providers (18.11)
internal/memory/schema.go — normalizes memory-provider-exported
tool schemas between bare and wrapped OpenAI formats. Handles
both function-call wrapper objects and bare name+description+params
shapes. Converges to a consistent internal representation.
2026-07-26 08:58:26 +10:00
sam 9d1a190908 chore: close memory provider verification issues 2026-07-26 02:27:06 +10:00
sam bff3c40ca9 merge: integrate memory infrastructure features (StreamingContextScrubber, schema normalization, built-in store, memory edit actions)
Resolve conflict in `.beads/` tracking files and scrubber implementation:
- Accept 8642f40's full implementation of StreamingContextScrubber with io.Writer interface, partial chunk boundary handling, nested tags, zero-alloc fast path, and comprehensive tests.
- Take incoming changes for issues tracking: close multiple verification tasks (18.10–18.13, 18.23–18.26) after adversarial review, update status for in-progress tasks (18.17, 18.18).
2026-07-26 02:26:25 +10:00
sam e5627930eb Merge pull request 'feat: StreamingContextScrubber for memory-context fence stripping (18.10)' (#77) from worktree-stateful-meandering-cloud into main
Reviewed-on: sam/archie-core#77
2026-07-25 16:15:41 +00:00
sam d48953d9e7 fix: Flush() leaking in-block content when stream ends mid-close-tag
Pending bytes buffered while inside a <memory> block were unconditionally
emitted as literal text by Flush, leaking hidden memory-context content
into visible streamed output when a stream cut off mid-tag (e.g. "...secret</mem").
Track pendingInBlock at capture time so Flush discards in-block pending
bytes instead of emitting them. Also corrects stale doc comments claiming
underlying writer errors are surfaced (they're intentionally discarded).

Found by adversarial review of 18.10.
2026-07-26 02:11:28 +10:00
sam 886591f674 feat: StreamingContextScrubber for memory-context fence stripping (18.10)
Stateful io.Writer that strips <memory>...</memory> fences from
streaming output, handling partial chunks split across tag
boundaries, nested tags, and missing closing tags.
2026-07-26 02:08:10 +10:00
sam f0c2a3eda5 beads: sync memory verification fixes + beads state 2026-07-26 01:57:41 +10:00
sam 0746950cb7 fix: MCP transport framing + crash retry + test harness (17.16)
framing.go: documented that readMessage caller must manage the
bufio.Reader lifetime. Creating a new bufio.Reader per call silently
drops buffered data and breaks multi-message streams.

transport.go: added crashCount check alongside startupFailures for
the MaxRetries limit — crash-looping subprocesses (spawn succeeds
but process immediately dies) now count toward the retry cap.

transport_test.go: replaced os.Exit(1) with syscall.Exit(1) in
crash-after-init behavior — Go's test harness intercepts os.Exit
and panics instead of exiting, keeping the process and its pipes
alive indefinitely. Replaced os.Exit(0) with runtime.Goexit() in
TestMCPServerHelper for clean goroutine exit.

Framing multiple-message test passes. Crash retry tests may need
further debugging in the transport spawn/reader lifecycle.
2026-07-26 01:57:41 +10:00
sam 3b9c280b12 wip: MCP transport types + framing + tests (17.16 partial)
Deepseek workflow ran before hitting session limit. Delivered:
  types.go — TransportState enum, JSON-RPC 2.0 Request/Response/Error
  framing.go — Content-Length header framing (LSP-compatible, 4MB max)
  transport_test.go — 585 lines of tests (red: implementation missing)

Memory verification fixes also included (lifecycle, manager, test updates).

transport.go still needed — the main MCPServer Connect/Disconnect
implementation. Tests are written, implementation is ~200 lines.
2026-07-26 01:40:05 +10:00
sam 156453cf74 fix: tool registry — 8 bugs from adversarial review (17.1, 17.2)
Verification review found 8 bugs, all fixed with regression tests:
  - Available() deadlock: CheckFn evaluated outside RLock
  - Register() shared references: Clone() defensive copy before storage
  - Clone() shallow nested schemas: deepCopyAny for maps/arrays
  - stringLit: no backtick/escape handling → strconv.Unquote
  - intLit: no negative/hex/octal/binary → strconv.ParseInt
  - Discover: abort on first error → Continue + errors.Join
  - isRegisterCall: too broad → only DefaultRegistry() calls match
  - schemaLit: only string values → handle ints, bools, nested maps

38 tests pass. All 34 packages green. golangci-lint + gofmt clean.

Closes: archie-core-abg.17.1, 17.2
2026-07-26 01:31:17 +10:00
sam bc1caf67aa feat: BasePlatformAdapter interface + Session types (16.3, 16.5, 16.9, 16.10, 16.11)
adapter.go — SendResult struct (success, message_id, error, retryable,
retry_after, continuation_message_ids, error_kind), ChatInfo struct
(name, type, chat_id, thread_id), BasePlatformAdapter interface
(Connect, Disconnect, Send, ChatInfo).

session.go — SessionSource (platform, chat_id/name/type, user_id/name,
thread_id, guild_id, scope_id, profile, is_bot, metadata) with Key()
for deterministic session key. SessionContext wrapping source with
connected_platforms, home_channels, session_key/id, timestamps.

session_store.go — SessionStore interface (GetOrCreate, Get, Reset,
List, GetByChannel, Touch, Delete, Close) + nellSessionStore backed
by nell.DocDB with mutex-guarded Save/Delete for race safety.

Verification: 4 findings fixed (gofmt alignment, race on mutex,
!= true anti-pattern, missing zero-value assertions). 58 tests pass
(incl -race), go vet + gofmt + golangci-lint clean.

Closes: archie-core-abg.16.3, 16.5, 16.9, 16.10, 16.11
2026-07-26 01:29:36 +10:00
sam 30b76f2ead feat: MessageType enum + MessageEvent struct (16.1, 16.2)
internal/gateway/messagetype.go — 14-value MessageType enum with
IsMedia(), IsRich(), IsEphemeral() classification helpers.

internal/gateway/messageevent.go — platform-neutral MessageEvent:
text, type, media attachments (file metadata, dimensions, duration),
reply context (message ID, text, author), editing support (edited
flag, edit timestamp), capability flags (streaming, code blocks).

Verification agent found 7 issues, 5 fixed (pointer types for
FileSize/Width/Height/Duration, MsgSticker in IsMedia, IsRich
coverage, ToLegacy edge cases), 2 correctly deferred.

42 tests pass. Build + vet + fmt clean.

Closes: archie-core-abg.16.1, archie-core-abg.16.2
2026-07-26 01:19:38 +10:00
sam b61b736f77 feat: tool registry package stub (archie-core-abg.17) 2026-07-26 01:08:02 +10:00
sam 3985f042c5 chore: commit beads state, gitignore local Dolt DB
.beads/issues.jsonl + .beads/interactions.jsonl are the sync
mechanism — they're passive exports committed to git and synced
via refs/dolt/data on the remote. The local Dolt database
(embeddeddolt/) is gitignored as binary state.

227 total issues, 101 ready to work.
2026-07-26 01:07:20 +10:00
sam 4c1cc151a5 feat: NellDB storage backend replacing SQLite (archie-core-abg.30)
Extracted store.TaskStore interface (20 methods) covering task
lifecycle, queries, and observability. Separate WorkflowStore for
the agent RPC proxy.

New internal/nell/ package:
  - OpenStore(path, nodeID) — persistent logstore.LogStore backend
  - OpenMemory(nodeID) — in-memory store for tests
  - Full TaskStore implementation via sdk.DocDB with MVCC

All consumers updated to interface:
  - daemon.Daemon.Store, webui.Server.Store, storerpc.Server.Store
  - workflow.TaskContext.Store (WorkflowStore)
  - cmd/archied uses nell.OpenStore(cfg.DBPath, cfg.BotUser)

7 integration tests. Build + 32 packages pass.

Closes: archie-core-abg.30
2026-07-26 01:06:05 +10:00
sam 54e21bfbde feat: feature-based YAML config with conf.d/ (LoadDir API)
Implemented archie-core-abg.19 — split monolithic config.toml into
modular feature files:

  config.yaml              — daemon-level fields
  conf.d/gateway.yaml      — chat channels, platforms
  conf.d/tools.yaml        — MCP servers, tool policy
  conf.d/memory.yaml       — memory provider config
  conf.d/models.yaml       — LLM providers and models
  conf.d/identities.yaml   — identity configs

New API: LoadDir(baseDir, overlayDir string) (Config, error)
  - Missing files = feature disabled (no error)
  - Legacy config.toml fallback when config.yaml absent
  - Overlay merges on top of base
  - Validation: unknown/duplicate/malformed files rejected
  - YAML tags added to all 14 config types
  - New types: MemoryConfig, ToolsConfig, MCPServer, ToolPolicy
  - Extra map[string]any for conf.d/ plugins

28 tests green. Build + full config suite passes.

Closes: archie-core-abg.19
2026-07-26 01:04:35 +10:00
sam 415065ac66 test: feature-based config contract (28 tests for archie-core-abg.19)
Defines the LoadDir(baseDir, overlayDir) (Config, error) API:
  - Daemon config in config.yaml
  - Feature files in conf.d/ (Linux convention): gateway, tools,
    memory, models, identities
  - Missing files = feature disabled (no error)
  - Legacy config.toml fallback when config.yaml absent
  - Overlay merges on top of base
  - Validation: unknown/duplicate/malformed files rejected
  - New types: MemoryConfig, ToolsConfig, MCPServer, ToolPolicy

28 red tests — awaiting implementer.
2026-07-26 00:54:26 +10:00
sam 25a45373d9 beads: add 91 adversarial verification tasks for quality gate
One verification bead per implementation task (91 total). Each blocks
its implementation — verification must pass before the task is done.

Verification acceptance criteria: independent adversarial review by
fresh agent, no dead code paths, no lint issues, tests pass, code
formatted, no hardcoded values in tests.

This enforces red-then-green + adversarial-verify for every
implementation across the full 78-task Hermes parity surface.
2026-07-26 00:15:33 +10:00
sam ddd61cc12f beads: decompose Hermes gap analysis into 78 sub-tasks
4 decomposition agents created 78 scoped, implementable sub-tasks:

  abg.16 Gateway:      15 tasks (P0: MessageType, MessageEvent, Adapter interface, SessionSource/Context/Store. P1: SendResult, optional methods, capability flags, dedup, media. P2: EphemeralReply, interactive UX, batching, truncation)
  abg.17 Tools:        20 tasks (P0: ToolEntry, registry, MCP stdio. P1: availability checks, result limits, sequential/concurrent dispatch, guardrails, progressive disclosure. P2: checkpointing, multimodal, MCP HTTP/SSE/sampling)
  abg.18 Memory:       16 tasks (P0: Provider interface, optional methods, lifecycle hooks, config, backup, Manager, scrubber, built-in tool. P1: sync pipeline, prefetch, write mirror, frozen snapshot, threat scanning, shutdown drain)
  abg.22 Security:      9 tasks (P2: PairingStore crypto/hash/expiry/atomic, auth mixin layered pipeline, env var allowlists, slash access policy, rate limiting)
  abg.24 Cron:          6 tasks (P2: job store, ticker, blueprints, lifecycle guard, provider interface, execution)
  abg.25 Operations:    5 tasks (P2: readiness probes, health endpoint, drain control, shutdown watchdog, systemd notify)
  abg.27 Webhook:       7 tasks (P2: HMAC validation, filters, scripts, templates, deliver_only, idempotency, MS Graph)

Total: 103 open beads representing the full Hermes feature surface.
2026-07-26 00:08:55 +10:00
sam 7c88248799 beads: sync issue state after Hermes gap analysis
15 new issues created under multi-agent-collaboration epic (abg.16-abg.30).
3 P0 epics: gateway model, tool registry, memory providers.
Delivery system scoped to NATS JetStream (no separate actor engine needed).
Session persistence scoped to NellDB (replaces SQLite+FTS5 with vector search).
2026-07-26 00:05:36 +10:00
sam 5396436d35 docs: Hermes feature gap analysis — 60 gaps across 11 subsystems
7 subagents mapped the full Hermes codebase (~200 .py files, ~500K LOC).
Master gap analysis identifies 60 missing features across:
  P0 (7): adapter lifecycle, message model, sessions, platform config,
         tool registry, MCP client, memory provider ABC
  P1 (13): send result, streaming, dedup, media, guardrails,
           progressive disclosure, session history, pairing/auth,
           CLI, config subcommands, profiles, readiness
  P2 (22): interactive UX, subagents, browser/search/file tools,
           compression, cron, dashboard, API server, delivery
  P3 (3): billing, OAuth, desktop app

15 new beads issues created under the abg epic. NellDB identified
as the persistent memory/config backend (P0 integration issue).
2026-07-26 00:03:42 +10:00
sam ae744d1cc4 feat: /spawn gateway command for chat-initiated tasks
TaskCreator interface on gateway.Router allows chat users to create
tasks via /spawn <title>. Implemented as gateway-local command (no LLM
needed). StoreTaskCreator wraps the daemon's store and uses the first
configured repo as the task workspace with timestamp-based synthetic
issue numbers to avoid collisions with Gitea-issued tasks.

Wired into archied's gateway startup: when repos are configured,
router.Tasks is populated so /spawn works. When no repos are
configured, /spawn returns 'not configured'.

Closes: archie-core-abg.13
2026-07-25 23:54:06 +10:00
sam 9df02f2b60 fix: golangci-lint baseline to zero findings (39→0)
19 files touched. All fixes are in code — no lint configuration
weakened:
- Added unchecked error handling in defer cleanup paths
- Removed dead variables and unused functions
- Added missing imports (io in gitea.go)
- Simplified conditions (TrimSuffix, De Morgan, slices.Contains)

Also fixes JetStream TempDir teardown race (archie-core-340):
t.Cleanup(srv.Shutdown) now registers after t.TempDir() so
server shutdown (including JetStream files) completes before
temp dir removal. Added srv.WaitForShutdown() for deterministic
cleanup ordering.

Closes: archie-core-756, archie-core-340
2026-07-25 23:51:13 +10:00
sam 399ca66e32 feat: multi-identity daemon config and core
internal/config/ — IdentityConfig struct for [[identities]] TOML
array. Backward-compatible: empty Identities uses legacy
single-identity fields unchanged. Validation for identity name,
bot_user, repos, forge type.

internal/daemon/ — IdentityRunner bundles per-identity forge client,
worktree manager, repos, and config. NewIdentityRunner constructor.
Run() starts one goroutine per identity with independent poll loops
for failure isolation. cycleForIdentity() polls identity repos via
dedicated forge client and drains shared store.

Design decision: per-identity goroutine with shared infrastructure
(store, NATS, container pool, event bus, web UI, plugin registry).

Closes: archie-core-abg.3, archie-core-abg.4
2026-07-25 23:50:58 +10:00
sam f36b6f2e41 feat: gateway architecture with Telegram channel
internal/gateway/ — Gateway interface, Router with gateway-local
command dispatch (commands handled directly, no LLM needed).
ModelManager interface for /model and /models support.

internal/channels/telegram/ — Telegram Gateway using go-telegram/bot
v1.18.0 (ported from GopherClaw). Webhook support, middleware,
allowed-chat filtering, long-message splitting.

internal/config/ — ChatConfig + TelegramConfig optional TOML block.

Cmd wiring in archied/main.go: starts Telegram gateway when
[chat.telegram] token_env is configured.

Gateway-local commands implemented: /status, /help, /model, /models.
Write actions (/spawn, /approve, /cancel) follow in abg.13.

Closes: archie-core-abg.5, archie-core-abg.14, archie-core-abg.15
2026-07-25 23:50:52 +10:00
sam e4c008af5e docs: multi-agent collaboration PRD from discovery interview
Discovery interview (archie-core-abg.1) confirmed scope: daemon-side
concurrency, cross-repo/config flexibility, agent-to-agent handoff,
chat channels with full pipeline parity, and general extensibility.
Design answer confirms per-identity goroutine with shared
infrastructure for failure isolation.
2026-07-25 23:50:45 +10:00
sam 8a6be134dc Merge pull request 'Feat/56 duplicate workflow warning fires on normal builtin overrides' (#76) from feat/56-duplicate-workflow-warning-fires-on-normal-builtin-overrides into main
Reviewed-on: sam/archie-core#76
2026-07-25 08:53:12 +00:00
sam 1975a3d842 Merge branch 'main' into feat/56-duplicate-workflow-warning-fires-on-normal-builtin-overrides 2026-07-25 08:53:02 +00:00
sam d5f3b5fe3d Merge pull request 'esc() misses single quotes, enabling attribute JS breakout in task row onclick (archie)' (#71) from feat/55-esc-misses-single-quotes-enabling-attribute-js-breakout-in-t into main
Reviewed-on: sam/archie-core#71
2026-07-25 01:30:09 +00:00
sam c603b3596d Merge branch 'main' into feat/55-esc-misses-single-quotes-enabling-attribute-js-breakout-in-t 2026-07-25 01:29:59 +00:00
sam a6c321475e Merge pull request 'metadata.archie.plugins frontmatter list is parsed but unused (archie)' (#72) from feat/50-metadataarchieplugins-frontmatter-list-is-parsed-but-unused into main
Reviewed-on: sam/archie-core#72
2026-07-25 01:29:52 +00:00
sam af9203bbd4 Merge branch 'main' into feat/50-metadataarchieplugins-frontmatter-list-is-parsed-but-unused 2026-07-25 01:29:41 +00:00
sam cec6dbcaf2 Merge pull request 'gofmt: fix misaligned TaskRef struct fields in storage.go (archie)' (#73) from fix/49-gofmt-fix-misaligned-taskref-struct-fields-in-storagego into main
Reviewed-on: sam/archie-core#73
2026-07-25 01:29:29 +00:00
sam 9c731710b3 Merge branch 'main' into fix/49-gofmt-fix-misaligned-taskref-struct-fields-in-storagego 2026-07-25 01:29:13 +00:00
sam fd0b60072e Merge pull request 'clip() can truncate strings mid-rune, producing invalid UTF-8 (archie)' (#74) from feat/47-clip-can-truncate-strings-mid-rune-producing-invalid-utf-8 into main
Reviewed-on: sam/archie-core#74
2026-07-25 01:28:47 +00:00
archie f83380e1bf clip() can truncate strings mid-rune, producing invalid UTF-8 (archie)
Implements #47
2026-07-24 18:06:37 +00:00
archie 6f72f38f5e fix: baseline gate repair (go vet ./...) 2026-07-24 18:04:15 +00:00
archie 8e76e5c7b3 fix: gofmt: fix misaligned TaskRef struct fields in storage.go (archie)
Fixes #49
2026-07-24 17:52:33 +00:00
archie af17f4cd66 test: failing repro for #49 2026-07-24 17:51:39 +00:00
archie ca92e24008 fix: baseline gate repair (go vet ./...) 2026-07-24 17:47:19 +00:00
archie 69b317a7c2 metadata.archie.plugins frontmatter list is parsed but unused (archie)
Implements #50
2026-07-24 17:42:17 +00:00
archie b4efcf84ba fix: baseline gate repair (go vet ./...) 2026-07-24 17:36:29 +00:00
archie 7632cba8e5 esc() misses single quotes, enabling attribute JS breakout in task row onclick (archie)
Implements #55
2026-07-24 17:33:22 +00:00
archie e4d1f728d7 fix: baseline gate repair (go vet ./...) 2026-07-24 17:28:19 +00:00
archie cc98d66bf2 Duplicate-workflow warning fires on normal builtin overrides too (archie)
Implements #56
2026-07-24 17:25:43 +00:00
archie c742e4243d fix: baseline gate repair (go vet ./...) 2026-07-24 17:12:17 +00:00
sam d4e4148b77 Merge pull request 'askpass helper rewritten on every git call despite "(once)" comment (archie)' (#68) from feat/59-askpass-helper-rewritten-on-every-git-call-despite-once-comm into main
Reviewed-on: http://git.catlow.cloud/sam/archie-core/pulls/68
2026-07-24 13:13:30 +00:00