Commit Graph
76 Commits
Author SHA1 Message Date
sam bb7b779b35 feat: refuse unrecoverable shell commands
Add internal/tools/command: an always-on floor under the shell tool,
enforced in the provider adapter so the lifted tau files stay diffable.

This is not a security boundary and does not pretend to be one. Anything
with a shell has countless ways past a rule list. What it buys is that
the failures with no recovery -- where you cannot intervene afterwards
because the machine, the filesystem, or the gateway itself is gone --
are not reachable by an ordinary mistake.

Rules: recursive delete of / and system directories, mkfs, raw block
device writes, fork bombs, kill -1, shutdown/reboot/init/systemctl
power, and sudo -S with no SUDO_PASSWORD configured. Two more are
archie's own: an agent must not stop the daemon that hosts /stop
(pkill/systemctl/docker), nor start a second one outside the supervisor.

The parsing is the actual work. Patterns are anchored to command
position, so `git commit -m "revert the rm -rf / change"` and
`echo "rm -rf /"` are ordinary commands, while `echo x; rm -rf /` and
`$(rm -rf /)` are not. Quoting, separators, ${VAR} braces, environment
assignments and privilege wrappers are all parsed rather than matched.

Probing the first version found two bypasses that made every rule
optional, both of which a model writes unprompted rather than only an
attacker: `bash -c "rm -rf /"` and `xargs rm -rf /` hide the command in
an argument. Segments are now followed into shell -c payloads, eval, and
command wrappers (xargs, timeout, nice, env, nohup, ...), bounded by a
nesting limit. Tests cover both directions -- the refusals and, at equal
length, the ordinary commands that must not trip them.
2026-07-29 18:45:43 +10:00
sam 9784cc6b3e fix: kill the whole process group when a shell command is cancelled
shell.go relied on exec.CommandContext's default cancellation, which
kills only the process it started. A shell's real work is a child of
that shell, so cancelling `go test ./...` reaped the shell and left the
test binary running, and `nohup x &` was never touched at all.

That made /stop a lie for the case it exists to cover: the turn
unwound, the user was told it stopped, and the command carried on.

Worse than expected, and caught by writing the test first: without a
process group the tool does not merely fail to kill the command, it
hangs. The backgrounded grandchild inherits stdout, so cmd.Wait blocks
until the shell timeout regardless of cancellation. Verified by
reverting the fix -- the new test fails on a 30s hang, not on the
survival assertion.

Run the command as its own process group, signal the group on
cancellation, and bound Wait with WaitDelay so an inherited pipe cannot
hold the turn open. setProcessGroup/signalProcessGroup were already
present in the lifted set, used by tau's agent.go but not its shell
tool; this extends them to the shell and clears the unused-code lint
that was flagging them.

Recorded as a mutation in shell.go's provenance header.
2026-07-29 18:36:22 +10:00
sam e03cef17c9 feat(tools): lift tau's file and shell tools into the tool registry
The chat agent had two tools -- memory_edit and skill_activate -- so it
could not read a file, search a tree or run a command. ai-sdk's runtime
already drives a tool loop for it (MaxSteps 8, toolExecute wired as the
executor), so the loop was never the missing piece: the registry was empty
of anything useful.

Lifted from tau internal/agent/tools at commit f5289ea, per the recorded
copy-and-mutate strategy: every file carries a provenance header naming its
tau source path, the commit, and the mutations made, so refreshing from
upstream is a diff rather than archaeology.

  internal/tools/builtin/          the lifted tools, ~3.1k lines
  internal/tools/provider/builtin/ the adapter to tools.ToolEntry

Six tools: read, write, edit, shell, grep, find. Tau's own tests came with
them and pass unmodified, which is the evidence that the lift is faithful.

Deliberately not lifted:
  - agent.go (1214 lines): subagent spawning, and the only real coupling in
    the package -- 10 tau packages including chat, sessions, store and
    registry. Nothing the chat agent needs to do work.
  - skill.go: depends on tau's skills package; archie has internal/skill and
    already exposes skill_activate.
  - docs.go: serves tau's own embedded documentation via tau/docs.

Not lifting the coordinator either. It is tau's agent loop (6.4k lines over
ten packages) and archie already has one in the ai-sdk runtime; porting it
would leave two loops competing. Its cancellation and tool events are worth
having, but they are a few hundred lines built against the existing path
rather than six thousand lifted onto a second one.

The adapter bridges two shape differences: archie hands handlers decoded
arguments where tau's executor takes raw JSON, and tau reports tool-level
failure in Result.IsError rather than as an error -- surfaced as a real
error so the runtime and the guardrail engine both see it. Tools are
classified idempotent or mutating for guardrails and budgets; that is
categorisation, not a jail.

chat.workspace roots the tools and defaults to empty, which registers none
of them. They read, write and execute, so the directory has to be a
deliberate choice rather than whatever the daemon started in -- in the
container that is /.

Verified: go build, go vet, gofmt clean; full suite passes.
2026-07-29 15:46:21 +10:00
sam 2d082eef89 Merge branch 'crew/lint6' 2026-07-29 10:37:52 +10:00
sam 712388dc6c fix: Phase 4 structural refactor — reduce lint findings from 28 to 5
cyclop (5→0): extract subcommand/case bodies in codesearch,
subprocess, Telegram, indexing
funlen (5→3): extract memory action handlers, tool provider startup
gocognit (7→2): extract config finalize/loadDirInto helpers, release
announcer, capture tool validation, Telegram gateway (200→108),
archie-agent main loop (35→0)
nestif (8→0): extract identity validation, container validation,
conf.d scanning, email multipart parsing, Telegram model/update
parsing, build-no-changes closure
interfacebloat (3→0): split Forge/IssueForge/PRForge/RepoForge,
SessionStore/SessionLifecycle/MessageHistory,
TaskStore/TaskLifecycle/TaskQueries/TaskEvents

Remaining: daemon process, workflow Feasibility/TDD/AgentStage,
archied main.go run (108→30 remaining). These require deeper
refactoring best done per-function in follow-up beads.

Updated contract tests and archied wiring tests to check both
main.go and the new telegram_setup.go.
2026-07-29 10:15:22 +10:00
sam a9e8cda4f2 fix(qa4.3): resolve nilerr, bodyclose, contextcheck, containedctx, fatcontext lint findings
- bodyclose: add deferred resp.Body.Close() in storage_test.go Docker test handler
- contextcheck: thread context through email handleSMTP/processMessage; replace
  context.Background() with context.WithoutCancel(ctx) in telegram, webhook, webui
  shutdown paths and container pool cleanup; add ctx parameter to Pool.Release
- containedctx: replace ctx context.Context struct field with stopCh chan struct{}
  in StdioTransport and SSETransport; thread lifecycle context through goroutine
  chains instead of storing it in structs
- fatcontext: lift context deadline derivation out of PrefetchContext loop
2026-07-29 10:10:42 +10:00
sam 4ee8579ddc refactor: phase 2 of lint remediation — mechanical fixes (qa4)
130 -> 96. forcetypeassert, predeclared and usetesting are now at zero.

Fanned out across four isolated worktrees partitioned by package so no
two workers could touch the same file: internal/tools (20 assertions),
internal/webui, internal/memory + internal/ratelimit, internal/secret.
Four stragglers outside those scopes fixed inline (internal/pairing,
internal/forge, internal/channels/telegram, internal/tools/mcp).

Unchecked type assertions became comma-ok with a t.Fatalf naming the
actual type. One site in a goroutine uses t.Errorf + return instead,
since t.Fatalf from a non-test goroutine is undefined behaviour.

predeclared renames chose names from meaning rather than adding a
suffix: max -> maxQueryLen, maxMatchLen, maxRequests, alphabetLen, and
nextBackoff's max -> ceiling. `real` -> `base` in the webui tests.

os.Setenv became t.Setenv, which restores on cleanup, so the paired
deferred Unsetenv calls went with it.

Note for future fan-outs: the agent worktrees branched from origin/main
(a7388a0), not from this branch, so every diff arrived based on a tree
without the phase 1 autofix. They were applied with `git apply --3way`
and the phase 1 changes to internal/memory/manager.go were verified to
survive — in particular the repaired `ctx, cancel =` assignment. A plain
`git apply` would have silently reverted it.
2026-07-28 01:37:20 +10:00
sam 15d746c2f6 style: phase 1 of lint remediation — autofix pass (qa4)
golangci-lint run --fix plus the formatters: errorlint, gocritic,
sloglint, usestdlibvars, usetesting, gci, gofumpt, goimports.
160 issues -> 130.

One autofix was wrong and had to be reverted by hand. gocritic rewrote

    ctx, cancel = context.WithTimeout(ctx, m.prefetchTimeout)

to `:=` inside an if-block in internal/memory/manager.go. That does not
compile, which is the good outcome — had it compiled, the shadowed ctx
would have scoped the timeout to the if-block and the prefetch deadline
would silently never have applied. Restored the assignment and commented
why it must not be a declaration.

Worth recording: --fix is not a safe no-op. It broke the build on a
24-file change, and the failure mode it was one compiler error away from
was a silent behaviour change.
2026-07-28 01:37:20 +10:00
sam c79c623e9c feat(chat): real system prompt for Archie — identity, tools, env (sc5)
The chat agent's entire system prompt was the four-sentence static string
in gateway.DefaultPersonas(). It carried no tool inventory, no environment
block and no instruction precedence, so the model invented its own
capabilities: asked to set something up it announced it "only has the
memory editing tool", then offered to walk through installing software it
has no tool to install.

Renders a template modelled on tau's agent.md.tpl + tau.agent.md:
instruction precedence, core rules, communication rules, a <tools> block
and an <env> block. The persona stays the style layer on top; the rules
and the tool inventory live in the template so switching persona cannot
remove them.

The tool list is derived from the same core.ToolSet handed to the model,
so the prompt cannot drift from what the agent can actually call, and an
empty toolset renders an explicit "no tools are available" rather than
silence the model fills in for itself.

Tool metadata is third-party data from MCP servers, so it is markup-escaped
(<, >, & only — not xml.EscapeText, which mangles prose into &#xA;/&#39;)
and whitespace-flattened, so a description cannot forge a closing tag or a
second "- tool: ..." list entry.
2026-07-28 01:37:19 +10:00
sam 0ff8729080 Merge branch 'main' into fix/53-handlesummary-ignores-errors-from-workflowstatsstagestatstok 2026-07-27 01:24:45 +00:00
sam ec8534dfe2 feat(gateway): ship model controls, shell tools, and release tracking 2026-07-27 06:16:39 +10:00
archie ef12eb6141 fix: baseline gate repair (go test ./... -count=1) 2026-07-26 20:05:07 +00:00
archie 5f964030c5 feat: add MCP client capabilities to archied and archie-agent (archie)
Implements #75
2026-07-26 19:51:36 +00:00
archie a7054250ff fix: baseline gate repair (go test ./... -count=1) 2026-07-26 19:41:12 +00:00
sam b9ecf37a48 refactor: replace fmt.Sprintf with fmt.Appendf in test 2026-07-27 04:03:23 +10:00
sam 6fe1a7d037 fix: lint cleanup across repo (errcheck, staticcheck, ineffassign, unused)
- daemon.go: real bug — maybeRetryParked resolved fg := d.forgeFor(task)
  (matching markDead's pattern) but called d.Forge.SetStateLabel instead
  of fg.SetStateLabel, silently using the wrong forge client for
  multi-identity/multi-forge task routing. Fixed to use fg throughout.
- telegram.go: updateThreadID existed but was never called anywhere;
  wired it into updateLoggingMiddleware so debug logs include thread_id.
- Two SA9003 empty-branch tests (nell adapter_test.go, tools
  toolentry_test.go) were dead assertions with no body; implemented the
  actual check instead of deleting them.
- memory manager_test.go: mockProvider.lastConfigSchema/lastBackupPaths
  were declared but never recorded; wired GetConfigSchema/BackupPaths to
  populate them and strengthened the aggregation tests to assert on
  per-provider content, not just counts.
- Removed genuinely dead gateway messageevent_test.go helpers (intPtr/
  i64Ptr, marked //go:fix inline, already fully superseded by the new()
  builtin used throughout the file).
- Remaining errcheck findings (defer Close/Flush/Write, os.Setenv, etc.
  in test files) fixed with the existing `_ = ` convention already used
  elsewhere in the codebase.

go build ./..., go test ./... (all packages ok), and
golangci-lint run ./... all clean.
2026-07-27 03:49:33 +10:00
sam 97e6a54da5 fix: MCP stdio transport lifecycle races and dead SendTimeout config
Implement the SendTimeout config field (previously silently ignored) via a
timeout-bounded write helper, and close two related races found during
adversarial review: Start()/Stop() could interleave across separate lock
acquisitions, and a subprocess spawned by a fresh Start() or a
crash-triggered restart could clobber a StateStopped already committed and
reported by a concurrent Stop(). Both spawn-commit paths now funnel through
a single guarded commitSpawnedProcess. Closes archie-core-abg.17.16,
archie-core-abg.17.23, archie-core-abg.17.41.
2026-07-27 03:46:47 +10:00
sam f73b2d1c8c feat: memory manager sync/prefetch/scan/shutdown + broad in-flight epic work
Checkpoint commit covering concurrent work across several archie-core-abg
epics to avoid losing in-progress state:

- Memory manager (18.7-18.16): background sync pipeline, prefetch with
  timeout, memory-write notification, threat scanning, shutdown drain
  with abandoned-write tracking, frozen snapshot pattern. Wired
  ScanContent and NotifyMemoryToolWrite into HandleToolCall so scanning
  and notification actually apply to writes, not just exist as unused
  primitives.
- Fixed a literal unresolved merge-conflict that had been committed to
  internal/memory/scrubber/scrubber.go.
- Tool provider/registry, MCP client/transport (stdio, HTTP, SSE),
  multimodal results, disclosure.
- Pairing device auth store, rate limiting.
- Gateway compression, session store, tasks.
- Various workflow, plugin host, and agentexec updates.
2026-07-27 03:34:46 +10:00
sam 8356863712 feat: Tools P1 — guardrails, classification, budget, availability, disclosure, dispatch
- ToolCallGuardrailConfig + GuardrailEngine with RecordSuccess/RecordFailure
- ToolClassification bitmask (idempotent/mutating) with dispatch gating
- TurnBudget for token/char accounting with spill-to-disk
- AvailabilityCache with TTL + failure grace window
- Progressive disclosure engine for tool schema filtering
- Tool dispatch executor with parallel/serial classification
- RecordSuccess fix: now returns GuardrailDecision (no-progress threshold)
- All tests pass with -race, 58 new test functions across 6 test files
2026-07-26 09:53:38 +10:00
sam f6d51416f0 chore: replace 607 em-dashes with double hyphens project-wide
Includes MCP crash-after-init test fix (crash moved from handle() to serve()
entry point so transport sees immediate subprocess death).
2026-07-26 09:53:33 +10:00
sam 0746950cb7 fix: MCP transport framing + crash retry + test harness (17.16)
framing.go: documented that readMessage caller must manage the
bufio.Reader lifetime. Creating a new bufio.Reader per call silently
drops buffered data and breaks multi-message streams.

transport.go: added crashCount check alongside startupFailures for
the MaxRetries limit — crash-looping subprocesses (spawn succeeds
but process immediately dies) now count toward the retry cap.

transport_test.go: replaced os.Exit(1) with syscall.Exit(1) in
crash-after-init behavior — Go's test harness intercepts os.Exit
and panics instead of exiting, keeping the process and its pipes
alive indefinitely. Replaced os.Exit(0) with runtime.Goexit() in
TestMCPServerHelper for clean goroutine exit.

Framing multiple-message test passes. Crash retry tests may need
further debugging in the transport spawn/reader lifecycle.
2026-07-26 01:57:41 +10:00
sam 3b9c280b12 wip: MCP transport types + framing + tests (17.16 partial)
Deepseek workflow ran before hitting session limit. Delivered:
  types.go — TransportState enum, JSON-RPC 2.0 Request/Response/Error
  framing.go — Content-Length header framing (LSP-compatible, 4MB max)
  transport_test.go — 585 lines of tests (red: implementation missing)

Memory verification fixes also included (lifecycle, manager, test updates).

transport.go still needed — the main MCPServer Connect/Disconnect
implementation. Tests are written, implementation is ~200 lines.
2026-07-26 01:40:05 +10:00
sam 156453cf74 fix: tool registry — 8 bugs from adversarial review (17.1, 17.2)
Verification review found 8 bugs, all fixed with regression tests:
  - Available() deadlock: CheckFn evaluated outside RLock
  - Register() shared references: Clone() defensive copy before storage
  - Clone() shallow nested schemas: deepCopyAny for maps/arrays
  - stringLit: no backtick/escape handling → strconv.Unquote
  - intLit: no negative/hex/octal/binary → strconv.ParseInt
  - Discover: abort on first error → Continue + errors.Join
  - isRegisterCall: too broad → only DefaultRegistry() calls match
  - schemaLit: only string values → handle ints, bools, nested maps

38 tests pass. All 34 packages green. golangci-lint + gofmt clean.

Closes: archie-core-abg.17.1, 17.2
2026-07-26 01:31:17 +10:00
sam bc1caf67aa feat: BasePlatformAdapter interface + Session types (16.3, 16.5, 16.9, 16.10, 16.11)
adapter.go — SendResult struct (success, message_id, error, retryable,
retry_after, continuation_message_ids, error_kind), ChatInfo struct
(name, type, chat_id, thread_id), BasePlatformAdapter interface
(Connect, Disconnect, Send, ChatInfo).

session.go — SessionSource (platform, chat_id/name/type, user_id/name,
thread_id, guild_id, scope_id, profile, is_bot, metadata) with Key()
for deterministic session key. SessionContext wrapping source with
connected_platforms, home_channels, session_key/id, timestamps.

session_store.go — SessionStore interface (GetOrCreate, Get, Reset,
List, GetByChannel, Touch, Delete, Close) + nellSessionStore backed
by nell.DocDB with mutex-guarded Save/Delete for race safety.

Verification: 4 findings fixed (gofmt alignment, race on mutex,
!= true anti-pattern, missing zero-value assertions). 58 tests pass
(incl -race), go vet + gofmt + golangci-lint clean.

Closes: archie-core-abg.16.3, 16.5, 16.9, 16.10, 16.11
2026-07-26 01:29:36 +10:00
sam 30b76f2ead feat: MessageType enum + MessageEvent struct (16.1, 16.2)
internal/gateway/messagetype.go — 14-value MessageType enum with
IsMedia(), IsRich(), IsEphemeral() classification helpers.

internal/gateway/messageevent.go — platform-neutral MessageEvent:
text, type, media attachments (file metadata, dimensions, duration),
reply context (message ID, text, author), editing support (edited
flag, edit timestamp), capability flags (streaming, code blocks).

Verification agent found 7 issues, 5 fixed (pointer types for
FileSize/Width/Height/Duration, MsgSticker in IsMedia, IsRich
coverage, ToLegacy edge cases), 2 correctly deferred.

42 tests pass. Build + vet + fmt clean.

Closes: archie-core-abg.16.1, archie-core-abg.16.2
2026-07-26 01:19:38 +10:00
sam b61b736f77 feat: tool registry package stub (archie-core-abg.17) 2026-07-26 01:08:02 +10:00