Commit Graph
1754 Commits
Author SHA1 Message Date
sam c79c623e9c feat(chat): real system prompt for Archie — identity, tools, env (sc5)
The chat agent's entire system prompt was the four-sentence static string
in gateway.DefaultPersonas(). It carried no tool inventory, no environment
block and no instruction precedence, so the model invented its own
capabilities: asked to set something up it announced it "only has the
memory editing tool", then offered to walk through installing software it
has no tool to install.

Renders a template modelled on tau's agent.md.tpl + tau.agent.md:
instruction precedence, core rules, communication rules, a <tools> block
and an <env> block. The persona stays the style layer on top; the rules
and the tool inventory live in the template so switching persona cannot
remove them.

The tool list is derived from the same core.ToolSet handed to the model,
so the prompt cannot drift from what the agent can actually call, and an
empty toolset renders an explicit "no tools are available" rather than
silence the model fills in for itself.

Tool metadata is third-party data from MCP servers, so it is markup-escaped
(<, >, & only — not xml.EscapeText, which mangles prose into &#xA;/&#39;)
and whitespace-flattened, so a description cannot forge a closing tag or a
second "- tool: ..." list entry.
2026-07-28 01:37:19 +10:00
sam 9e5fd7c38b Merge branch 'main' into fix/43-loaddir-no-timeout-around-yaegi-eval-can-hang-daemon-startup 2026-07-27 08:18:09 +00:00
sam a7388a0a59 Merge pull request 'Fetch() masks JetStream errors as empty-queue when zero msgs arrive (archie)' (#84) from fix/35-fetch-masks-jetstream-errors-as-empty-queue-when-zero-msgs-a into main
Reviewed-on: sam/archie-core#84
2026-07-27 08:17:51 +00:00
archie 5045592d41 fix: Fetch() masks JetStream errors as empty-queue when zero msgs arrive (archie)
Fixes #35
2026-07-27 04:37:17 +00:00
archie 03899171e6 test: failing repro for #35 2026-07-27 04:36:14 +00:00
archie 7c294f88f7 fix: LoadDir: no timeout around Yaegi Eval can hang daemon startup (archie)
Fixes #43
2026-07-27 03:04:28 +00:00
sam bcca6f9c25 feat(telegram): add approved component updates 2026-07-27 13:02:11 +10:00
archie 255f580cf6 test: failing repro for #43 2026-07-27 03:01:56 +00:00
sam c1c3a973c6 feat(telegram): report installed component versions 2026-07-27 12:53:59 +10:00
archie ce0fc1fac0 fix: skillscript.Run ignores context, cannot be canceled/timed out (archie)
Fixes #45
2026-07-27 02:41:44 +00:00
archie c3aa0a0a02 test: failing repro for #45 2026-07-27 02:35:35 +00:00
archie febf70a754 fix: Transition() ignores from status guard, allowing lost updates (archie)
Fixes #48
2026-07-27 02:26:52 +00:00
archie e94a418c99 test: failing repro for #48 2026-07-27 02:24:04 +00:00
archie 9ac5817b84 No test coverage for StageStats query (archie)
Implements #51
2026-07-27 02:20:39 +00:00
archie 79f82101ee fix: handleSSE silently swallows EventsSince backlog fetch errors (archie)
Fixes #52
2026-07-27 02:17:32 +00:00
archie 7927eab07b test: failing repro for #52 2026-07-27 02:12:59 +00:00
sam 647537c00a refactor(telegram): keep one model selector command 2026-07-27 11:44:27 +10:00
sam 2facb2be1b Merge pull request 'handleSummary ignores errors from WorkflowStats/StageStats/TokensByDay (archie)' (#78) from fix/53-handlesummary-ignores-errors-from-workflowstatsstagestatstok into main
Reviewed-on: sam/archie-core#78
2026-07-27 01:24:57 +00:00
sam 0ff8729080 Merge branch 'main' into fix/53-handlesummary-ignores-errors-from-workflowstatsstagestatstok 2026-07-27 01:24:45 +00:00
sam 329776de9c feat: establish Archie identity and use stable callback tokens 2026-07-27 11:03:40 +10:00
sam ec8534dfe2 feat(gateway): ship model controls, shell tools, and release tracking 2026-07-27 06:16:39 +10:00
archie 0b8560c2c2 fix: handleSummary ignores errors from WorkflowStats/StageStats/TokensByDay (archie)
Fixes #53
2026-07-26 20:08:20 +00:00
archie 902fd28bf5 test: failing repro for #53 2026-07-26 20:07:32 +00:00
archie ef12eb6141 fix: baseline gate repair (go test ./... -count=1) 2026-07-26 20:05:07 +00:00
archie 5f964030c5 feat: add MCP client capabilities to archied and archie-agent (archie)
Implements #75
2026-07-26 19:51:36 +00:00
archie a7054250ff fix: baseline gate repair (go test ./... -count=1) 2026-07-26 19:41:12 +00:00
sam 3cab289b41 refactor: centralize command specs and dynamic help text for Telegram 2026-07-27 05:34:46 +10:00
sam 19a6311f6c feat(telegram): /restart to reload config without dropping tasks (16.35)
Operator escape hatch: reload gateway config (notably the sender
allowlist) or recover a wedged adapter from chat, with no shell access to
the host.

Start becomes a supervisor loop that relaunches the bot on request, so a
restart is scoped to the gateway: the daemon keeps running and in-flight
agent tasks are untouched. Exiting the process would also have restarted
under the unless-stopped policy, but would have killed running work.

Two failure modes are handled deliberately:

- The handler runs on the very bot the supervisor is about to stop, so it
  hands off over a buffered channel and returns instead of tearing down
  inline, which would deadlock the restart it just requested. A duplicate
  request while one is in flight is dropped, not queued.
- A failed config reload logs and relaunches on the previous settings, so
  a bad config edit cannot take chat down permanently -- which would also
  remove the means to fix it.

Authorisation is inherited from authorizedMessage: only allowlisted
senders reach any handler, so /restart is admin-only by construction.
main.go supplies Reload, which re-reads config from disk and refreshes
token and allowlist; other fields still need a full daemon restart, since
they are wired at construction.
2026-07-27 05:21:58 +10:00
sam 9fd4696492 fix(telegram): publish command menu to all scopes, not just default
The menu still showed ~60 stale hermes commands (/steer, /reload_mcp,
/background) even though SetMyCommands succeeded and logged 8 commands.

Telegram resolves a chat's menu by scope specificity rather than by
recency: a list registered against all_private_chats wins over the
default scope in every DM. We wrote only the default scope, so hermes's
older per-scope registrations kept shadowing ours. getMyCommands confirms
it -- default held our 8, all_private_chats and all_group_chats each held
hermes's ~60.

That state lives on Telegram's side, keyed to the bot token, so it
survived the migration off hermes and would survive any redeploy. Publish
to default, all_private_chats and all_group_chats so the leftovers are
overwritten rather than shadowing us. One failing scope no longer aborts
the rest.
2026-07-27 05:18:26 +10:00
sam cb147b7a37 feat(telegram): deny-by-default sender allowlist + command menu (16.33, 16.34)
Allowlist (16.33). telegram.New was called with a nil allowlist and
isChatAllowed returned true whenever the list was empty, so any Telegram
user who found the bot could talk to it; AllowedChatIDs was not plumbed
into config, so there was no way to close it. Chat tools run with the
daemon's own authority, so this had to be fixed before wiring any real
tools to the chat agent.

Now chat.telegram.allowed_user_ids configures it and an empty list denies
everyone: a bot handle is public, so failing open is not a safe default.
The check matches the sender (from.id) rather than the chat, so adding
the bot to a group cannot be used to inherit access. Rejections log the
sender id and answer once, so a legitimate user missing from the list can
tell the bot is misconfigured rather than dead. Startup warns when the
gateway is enabled with an empty list.

The previous TestIsChatAllowed asserted 'empty allowlist allows all' --
it encoded the vulnerability -- and is replaced by fail-closed tests.

Command menu (16.34). SetMyCommands was never called, so Telegram showed
no menu and the commands Router implements were undiscoverable; asked
about /commands the LLM confidently answered it had none. The published
list mirrors what Router actually handles. Failure is logged, not fatal:
it costs discoverability, not function.
2026-07-27 05:14:34 +10:00
sam 285c618f74 fix(chat): drain FullStream, not TextStream — streaming deadlocked every reply
Streamed chat replies hung on the first token: the draft rendered an
empty cursor, the typing indicator never cleared, and the turn never
completed.

RouteStream consumed stream.TextStream, but ai-sdk's channel contract
makes FullStream authoritative and requires draining it until close --
its writes are synchronous, so an unread FullStream applies backpressure
and stalls the producer. Once its buffer filled, the producer blocked,
TextStream stopped yielding, and the range loop blocked forever.

TextStream was the wrong source regardless: it is explicitly best-effort
and drops deltas when not drained, so a reply reconstructed from it can
silently lose text. Range over FullStream and collect StreamPartTextDelta
parts, exactly as the SDK documents.

Also bumps ai-sdk v0.1.17 -> v0.1.18 (build, full suite and lint clean).
2026-07-27 05:04:04 +10:00
sam 3e3911a423 feat(telegram): native rich messages + streamed replies (16.32)
Upgrades go-telegram/bot v1.18.0 -> v1.22.0 and uses the Bot API's rich
message support instead of hand-rolled formatting.

The previous commit converted LLM Markdown into Telegram's HTML subset,
on the premise that the API could not take Markdown directly and had no
table element. Both are true of v1.18.0 but not of v1.22.0, which adds
sendRichMessage/sendRichMessageDraft taking InputRichMessage{Markdown},
plus RichBlockTableCell for native tables. markdownToHTML and its tests
are deleted: LLM Markdown, tables included, now goes to Telegram as-is.

Replies also stream. runtime.ChatStream already returned a TextStream
and sendRichMessageDraft renders partial output, so the only missing
piece was a gateway seam: gateway.LLMStreamResponder plus Router.
RouteStream, which degrades to the blocking Route when unset so any
adapter can call it unconditionally. Local commands answer from local
state and are not streamed; an unrecognised /command is, since Route
hands it to the LLM. main.go derives both responders from one shared
turn implementation.

Draft updates are throttled to one per 700ms to respect per-chat rate
limits, and every streaming/rich path falls back rather than dropping a
reply: a rejected rich send retries unformatted, and a failed draft
update is skipped since the final message is authoritative.
2026-07-27 04:57:36 +10:00
sam b52accf24a config: set containers.network explicitly for the docker deployment
Pins the agent-container network instead of relying on selfNetwork()
auto-detection, which silently fell back to the default bridge and broke
NATS resolution for spawned agents.
2026-07-27 04:51:10 +10:00
sam bf0e348cbb fix: pass HEYARCHIE_TELEGRAM_BOT_TOKEN through to archied
The telegram chat gateway is enabled via chat.telegram.token_env in
config.toml, but compose never forwarded the named variable, so
cfg.Chat.Telegram.TokenEnv resolved to empty and archied exited 1.
Same class of bug as ARCHIE_GITEA_TOKEN_ARCHIE.
2026-07-27 04:51:10 +10:00
sam cb1b0eb11a feat(telegram): typing indicator + Markdown rendering (part of 16.6)
Two gaps visible in the live heyarchie bot:

1. No typing indicator. A non-command message runs a full LLM turn
   (seconds), during which the chat looked dead. defaultHandler now
   shows the typing action for the duration, refreshed every 4s since
   Telegram clears it after ~5s.

2. ParseMode was never set, so LLM Markdown reached users verbatim:
   literal **bold**, backticks, and pipe tables. helpHandler was worse
   -- it emitted <b>Archie Gateway</b> with no ParseMode, so the raw
   tags were shown.

The SDK has no Markdown->Telegram converter (EscapeMarkdown does the
inverse: it escapes text so it renders literally), and neither native
parse mode helps, since Telegram's Markdown dialects use single-asterisk
*bold* whereas LLMs emit **bold**. So markdownToHTML converts to the
HTML subset Telegram accepts, mapping headings to <b>, bullets to '•',
and tables to <pre> (Telegram has no table element). Send falls back to
unformatted text if Telegram rejects the markup, so a conversion bug can
never swallow a reply.
2026-07-27 04:47:21 +10:00
sam 18948ac0e3 chore: remove accidentally-committed build/test scratch artifacts
.gotmp/ (Go build cache and test temp-dir output, created as a
workaround for a tmpfs disk-quota issue in this environment) got swept
into a prior commit by a broad `git add -A`. Remove it and gitignore
the directory so this doesn't recur.
2026-07-27 04:43:37 +10:00
sam 325aa7ae35 test message 2026-07-27 04:39:39 +10:00
sam 4c0fc12450 test message 2026-07-27 04:39:27 +10:00
sam 52015c81b0 fix: pass ARCHIE_GITEA_TOKEN_ARCHIE through to archied container
The archied service only forwarded ARCHIE_GITEA_TOKEN, but config.toml
names ARCHIE_GITEA_TOKEN_ARCHIE for the archie identity. Recreating the
container via docker compose (rather than a manual docker run with an
explicit -e) silently dropped the token.
2026-07-27 04:31:43 +10:00
sam 38a746a891 fix: make agent-container Docker network explicitly configurable
archied's container-mode agent tasks were all failing with "nats: no
responders available for request" in production. Root cause: pool.go's
selfNetwork() auto-detects the daemon's own Docker network by looking up
a container named after os.Hostname(), then joins spawned agent
containers to that same network so they can resolve "nats" by name.
On carina this detection was silently failing (wrong/stale hostname vs.
actual container ID) and returning "", so every per-task agent container
landed on the default bridge network instead — unable to resolve "nats",
never subscribing, hence "no responders" on every request.

selfNetwork() swallowed every failure silently (only logged on success),
so this had no diagnostic trail.

Fix: added Containers.Network config field (containers.network in
config.toml) so deployments can name the network explicitly instead of
relying on fragile self-detection; selfNetwork() remains as a fallback
but now logs a warning on every failure path so this doesn't silently
recur. Wired through container.Config and cmd/archied/main.go.
2026-07-27 04:25:28 +10:00
sam 63719cc3ec fix: restore [forge] token_env backward compat, fixing prod crash-loop
Both archied production containers (archie-core-archied-1 and -winter-1)
were crash-looping on carina with "forge token (engine \"env\", key
\"ARCHIE_GITHUB_TOKEN\") is required" despite being configured for gitea.

Root cause: the secrets-engine migration replaced [forge]'s flat
token_env string field with a {engine, key} struct (Forge.Token), but
dropped the old field entirely instead of keeping it as a deprecated
fallback like NATSConfig and TelegramConfig already do. TOML silently
ignores unknown keys, so deployed configs still using token_env had
their token config dropped without any warning, and finalize()
defaulted to demanding ARCHIE_GITHUB_TOKEN.

Fix: re-added Forge.TokenEnv and honor it in finalize() when the new
Token struct is unset, matching the existing NATS/Telegram pattern.
New-style [forge.token] still wins if both are present. Added
regression tests for both cases.

Operationally also fixed the two live configs on carina
(~/.config/archie/config.toml, winter.toml) to the new schema, and
added missing work_dir/db_path to winter.toml (its container had no
config-overlay, so it was defaulting to /root/.local/share/archie
instead of the mounted volume). Both containers verified healthy and
processing real issues after restart.
2026-07-27 04:13:04 +10:00
sam b9ecf37a48 refactor: replace fmt.Sprintf with fmt.Appendf in test 2026-07-27 04:03:23 +10:00
sam 80cd473559 fix: remove local filesystem replace directive for NellDB
go.mod pointed NellDB at a local dev-machine path
(/work/apps/nell-engine) instead of the published v0.2.5 module,
which the Gitea CI runner can't resolve — every push since the
replace was added has failed the deploy workflow's docker build
(go mod download: no such file or directory), silently leaving
carina on a two-day-old image. Confirmed nell-engine's local HEAD
is functionally identical to the already-tagged-and-pushed v0.2.5
(only a trivial .gitignore commit ahead), so dropping the replace
and re-resolving from the registry is a no-op for behavior.
2026-07-27 04:00:51 +10:00
sam 6fe1a7d037 fix: lint cleanup across repo (errcheck, staticcheck, ineffassign, unused)
- daemon.go: real bug — maybeRetryParked resolved fg := d.forgeFor(task)
  (matching markDead's pattern) but called d.Forge.SetStateLabel instead
  of fg.SetStateLabel, silently using the wrong forge client for
  multi-identity/multi-forge task routing. Fixed to use fg throughout.
- telegram.go: updateThreadID existed but was never called anywhere;
  wired it into updateLoggingMiddleware so debug logs include thread_id.
- Two SA9003 empty-branch tests (nell adapter_test.go, tools
  toolentry_test.go) were dead assertions with no body; implemented the
  actual check instead of deleting them.
- memory manager_test.go: mockProvider.lastConfigSchema/lastBackupPaths
  were declared but never recorded; wired GetConfigSchema/BackupPaths to
  populate them and strengthened the aggregation tests to assert on
  per-provider content, not just counts.
- Removed genuinely dead gateway messageevent_test.go helpers (intPtr/
  i64Ptr, marked //go:fix inline, already fully superseded by the new()
  builtin used throughout the file).
- Remaining errcheck findings (defer Close/Flush/Write, os.Setenv, etc.
  in test files) fixed with the existing `_ = ` convention already used
  elsewhere in the codebase.

go build ./..., go test ./... (all packages ok), and
golangci-lint run ./... all clean.
2026-07-27 03:49:33 +10:00
sam 64352a06f3 beads: sync export after commit 2026-07-27 03:46:56 +10:00
sam 97e6a54da5 fix: MCP stdio transport lifecycle races and dead SendTimeout config
Implement the SendTimeout config field (previously silently ignored) via a
timeout-bounded write helper, and close two related races found during
adversarial review: Start()/Stop() could interleave across separate lock
acquisitions, and a subprocess spawned by a fresh Start() or a
crash-triggered restart could clobber a StateStopped already committed and
reported by a concurrent Stop(). Both spawn-commit paths now funnel through
a single guarded commitSpawnedProcess. Closes archie-core-abg.17.16,
archie-core-abg.17.23, archie-core-abg.17.41.
2026-07-27 03:46:47 +10:00
sam d227330243 beads: sync export after commit 2026-07-27 03:34:53 +10:00
sam f73b2d1c8c feat: memory manager sync/prefetch/scan/shutdown + broad in-flight epic work
Checkpoint commit covering concurrent work across several archie-core-abg
epics to avoid losing in-progress state:

- Memory manager (18.7-18.16): background sync pipeline, prefetch with
  timeout, memory-write notification, threat scanning, shutdown drain
  with abandoned-write tracking, frozen snapshot pattern. Wired
  ScanContent and NotifyMemoryToolWrite into HandleToolCall so scanning
  and notification actually apply to writes, not just exist as unused
  primitives.
- Fixed a literal unresolved merge-conflict that had been committed to
  internal/memory/scrubber/scrubber.go.
- Tool provider/registry, MCP client/transport (stdio, HTTP, SSE),
  multimodal results, disclosure.
- Pairing device auth store, rate limiting.
- Gateway compression, session store, tasks.
- Various workflow, plugin host, and agentexec updates.
2026-07-27 03:34:46 +10:00
sam 2b1045c0ad feat: secrets-engine registry with yaegi extension interface (archie-core-xec)
Adds internal/secret: a pluggable Engine interface (Name/Version/Resolve)
resolved through a Registry, so secrets backends (sops, Bitwarden Secrets
Manager, Vault/OpenBao) aren't hard-coded into the binary. Only the "env"
engine is compiled in; custom engines are Yaegi-interpreted .go plugins
loaded via Registry.LoadDir, mirroring internal/plugin's conventions
(secretextract provides the generated, nil-guarded symbol table for the
Yaegi/Go interface boundary).

config.Forge.TokenEnv is replaced end-to-end with config.Forge.Token
(secret.SecretRef{Engine, Key}) -- finalize() defaulting, forge token
resolution in cmd/archied/main.go, and per-identity validation that now
requires an explicit token instead of silently resolving to "". No
backwards-compat shim.

Provider.APIKeyEnv, NATSConfig.TokenEnv, and TelegramConfig.TokenEnv are
deliberately not migrated here -- they're forwarded by name to
ai-sdk/runtime and subprocess workers rather than resolved centrally,
which needs its own eager-resolve-then-inject design pass
(archie-core-xec.4). The three shipped engine plugins (sops, bws,
vault/openbao) are tracked separately as archie-core-xec.1-3.

Adversarially reviewed by a fresh agent with no implementation context;
fixed the two findings that survived (missing per-identity token
validation, underdocumented nil-wrapper Resolve edge case).
2026-07-26 14:49:32 +10:00
sam 146acf5466 feat: secret engine registry foundation (archie-core-xec)
internal/secret package:
- Engine interface (Name, Version, Resolve) for pluggable secret backends
- SecretRef type replacing TokenEnv/APIKeyEnv string fields
- ResolveOrEnv for backward-compatible env-var fallback
- Built-in env engine registered by default
- Registry with Register/Get/Resolve/LoadDir (yaegi-based)
- DefaultRegistry() singleton for one-off resolution
- 12 tests, race-clean

Scoped to foundation only. Provider/NATS/Telegram migration deferred
to xec.4 (separate bead — those call sites forward env var names to
subprocesses/runtimes and need eager-resolve-then-inject, not a simple
field rename).

40/40 packages pass with -race.
2026-07-26 14:20:19 +10:00