Commit Graph
19 Commits
Author SHA1 Message Date
sam 440d2e8e46 fix: reject values that exceed Int32 for all SQL LIMIT params 2026-09-24 21:45:27 +10:00
sam c70fb61bfc fix(gateway): stamp the session platform from the channel that carried the message
One Router serves every channel and named itself "web", so Source.Platform -- the
FIRST component of the session natural key, and the platform GetByChannel filters
on -- was a constant, true of no channel but the dashboard. Two consequences
followed: the per-user identity policy was real code that no production call site
reached with a real channel name, and a curator pass's write to agent-user memory
could not be read back by the turn that wrote it, because that turn resolved no
UserID.

The channel now travels with the message. Frontends name the channel they carry
(pb.Message.platform: telegram, email, webhook, web), and the Gateway stamps the
session from it, falling back to its own name when a sender names none -- which is
what every existing row holds and what a frontend that does not participate keeps
doing.

Identity resolution reads the platform off the inbound rather than closing over a
channel name, so carriesPersonIdentity finally decides something in production:
telegram and email resolve a person, the dashboard and a webhook resolve nothing,
and an unknown platform fails closed.

Deliberately NOT changed, because it is where this could quietly widen scope:
TurnRunnerConfig.Channel still keys tool availability, so no channel gains or
loses a tool here. Only session lookup and identity follow the message's platform.

Consequence the maintainer accepted: a chat whose active session came from /new or
/branch re-resolves once, since GetByChannel now filters on the channel's name. A
chat with no /new re-resolves onto the same deterministic session id and heals
silently.

Verified: gateway, archied, gatewayrpc, messaging, webui, channels and cmd suites,
go vet ./..., go build ./..., golangci-lint run ./... (0 issues), docs:check.
New contracts: TestResolveSessionKeyStampsTheMessagesPlatform, the per-platform
identity table, and the transport-context wire round trip.
2026-09-22 04:30:15 +10:00
sam ae848fd307 feat(gateway): compress a chat session when a turn crosses its context budget 2026-09-22 03:35:55 +10:00
sam e3b1a859a8 feat(gateway): flip channel boundary onto messaging.Message
Replaced gateway.Message as the currency at the channel-facing boundary
with the canonical internal/domain/messaging.Message. Channel adapters
now build the record directly (RoleUser, ConversationID, Sender) and the
Router hands it back unchanged, so nothing converts between two shapes in
process. gateway.Message is deleted; the dashboard route that had no home
in the record lives in a new Inbound envelope (Message + transport-only
Page), which reaches the system prompt and is never persisted.

Role is now set at construction instead of derived at the store boundary,
and the assistant reply is a direct messaging.Message (RoleAssistant,
Sender=botUser). The gRPC proto is unchanged: pb.Message still carries no
role, and both sides keep deriving it from the owning session's bot
identity via gateway.RoleForSender (ToStoredMessage/FromStoredMessage are
deleted).

Adds tests that the old shape could not express: Page reaches the system
prompt but not the persisted record, and a channel-built Inbound round-trips
with the Role the channel set.

Refs: archie-core-9g5e
2026-09-09 01:05:36 +10:00
sam 89d8dc0e7f feat(messaging): migrate gateway session persistence onto domain types
SessionStore message history now stores/loads domain/messaging.Message:
sender attribution, derived role, and conversation address are canonical
record fields. SQLite gains a role column with open-time backfill for
pre-migration databases. gateway.Message remains the channel-facing
transport; conversion lives in one boundary module. ChatContract,
gatewayrpc proto, and dashboard JSON are unchanged; the RPC client
splits into ChatContract (Client) and SessionStore (StoreClient) views.

Sessions stay SessionContext: UUID keys and platform/bot/title/branch
metadata have no lossless home in Conversation; keyed migration tracked
separately once Agent/user/binding ownership settles.

Refs: archie-core-d9dv
2026-09-08 21:28:19 +10:00
sam 77b1eaa2df fix(gateway): make UUIDv7 session references unambiguous 2026-08-25 03:28:38 +10:00
sam 9e89e0cd9e fix(chat): restore transcript and update replay 2026-08-25 03:05:28 +10:00
sam 61b4b7b0cf refactor(store): replace legacy persistence with SQLite
Task and event persistence now uses a dedicated SQLite sidecar derived from the configured state prefix, matching the independently persisted conversation store. The configured path now means a state-path prefix rather than a database file so fresh SQLite state never attempts to interpret the previous storage format.

Remove the retired implementation and dependency completely, strengthen SQLite migrations and concurrency behavior, restore backend-neutral coverage, and align configuration, operational guidance, UI labels, and deployment documentation with the new storage and host-daemon topology.
2026-08-09 22:27:25 +10:00
sam 1081a6c3ed feat(daemon): identity-scoped isolation + gateway session titles and /delete
Multi-identity daemon isolation:
- Split runIdentities into per-identity poll loops plus one shared
  maintenance-and-drain loop, so reconcilePRs and storage cleanup run in
  multi-identity mode and the global max_concurrency / per-repo
  serialization holds across identities.
- Make the poll path identity-aware: each identity polls with its own
  forge client, bot user, label, trigger, and ack reaction.
- Make allowConcurrentFor task-based and identity-aware via repoFor.
- Roll back storage mounts when container acquisition fails; log
  Store.Update failures after branch set.

Container-mode RPC identity routing:
- forgerpc/worktreerpc gain identity-scoped subjects (SubjectFor,
  RegisterFor) and identity-carrying clients.
- registerTaskRPCServers registers one server pair per identity (plus
  root), so a container-mode task owned by a non-root identity is served
  by its own forge client and worktree manager.

Gateway/Telegram:
- LLM-proposed session titles generated in the background after turns
  (TitleGenerator + chatTitleGenerator), persisted by the gateway.
- /delete session command with prefix guard and tracker forget.
- Shared session display helpers (shortSessionID, sessionDisplayTitle,
  resolveSessionRef).
2026-08-08 01:32:35 +10:00
sam 308666b410 chore: remove legacy terminology and dead code
- Reword docs and comments to drop vague "legacy" qualifiers where they no longer describe deprecation status
- Remove unused code: RecentEvents, MessageEvent.ToLegacy, index setState, stderr redaction tool, telegramOnlyCommands
- Delete or comment out tests covering removed or superseded behavior
2026-08-06 22:55:26 +10:00
sam fa6f4aef5f fix(gateway,daemon): repair regressions found by the adversarial review
Six defects, four of them introduced by the fixes in dbacb96, 44d43ef,
d5e016e, de1a899 and 4d44ef7. This is the third round in this codebase
where a round of fixes broke something the next round caught.

SANDBOX SILENTLY DROPPED. startContainers degrades a failed container
pool to nil, and the daemon's only test was `if ContainerPool != nil`.
So `containers.enabled = true` plus a missing agent image ran every
task's agent loop in-process, with the daemon's own authority and the
host filesystem -- the one thing [containers] exists to prevent -- while
logging only that containers were disabled. Nil now means two things and
they are distinguished: SandboxRequired parks the task instead. The
daemon still stays up; chat and the dashboard are unaffected.

SUMMARY AT THE WRONG END OF HISTORY. compressedHistory returns [head,
summary, tail], but ordering is by timestamp, not slice position. The
retained messages keep their original timestamps -- that is what
preserves their identity -- so the summary, being new, was clamped to
now and sorted after the tail it was meant to precede. The model read
the recent conversation and was only then told earlier context had been
summarised. Confirmed on both backends: index 23 of 24.

The monotonic clamp belongs to the append path, where an inbound message
must not sort ahead of the one that provoked it. ReplaceMessages is a
rewrite primitive and now honours explicit placement, and the summary
takes a timestamp inside the span it replaces.

A COMPRESSION SUMMARY ANSWERED A QUESTION. Following from the above,
PriorReply saw a bot-attributed record immediately after an unanswered
message and replayed it as the answer: a redelivered question was
"answered" with the compression banner and the model never ran. The
user's question was silently swallowed.

RECENCY STILL DIVERGED. sessionRecency preferred LastActiveAt whenever
set; SQLite used MAX(last_active_at, created_at). Those agree only when
LastActiveAt is the later of the two, so a clock step or an older record
made the caller and the store order the same sessions differently.

LASTACTIVEAT WAS NEVER UPDATED. Touch had no caller, so "resolve the
most recently active session" meant the most recently created one -- an
operator who moved to an older session with /topic and worked there was
dropped back into the newer, abandoned one on restart, which is the bug
d5e016e claimed to fix, with the direction reversed.

/branch WITH NO ACTIVE SESSION PANICKED, slicing an empty parent ID for
its default title. Pre-existing, but reachable: email and webhook call
Route directly without resolving first, and the email path runs in a
goroutine with no recover, so one message took the daemon down.

The sandbox guard has no test. The honest reason is that exercising it
needs a daemon with a forge and worktree the current harness does not
build, and a test that re-states the condition proves nothing -- which
is the same false confidence that let these regressions through.
2026-08-05 04:24:16 +10:00
sam a1a111ff35 fix(gateway): give branch-inherited messages their own identity
/branch copies the parent's history into the child. Because the store
honours an incoming MessageID -- which is what lets a read-modify-write
preserve identities -- the copies kept the parent's canonical IDs, so
two sessions claimed one identity and branch-point correlation was
ambiguous. Two branches from one parent made it three. It also broke
dedup in the child: an inherited message carried an ID derived from the
parent's session, so redelivering it appended (confirmed on both
backends: 3 messages became 4).

newMessageID's doc already claimed the opposite behaviour, and the
Message.MessageID doc said it was "ignored on write" -- which is exactly
the misreading that produced this. Both now describe what the code does,
including the obligation on a caller copying messages between sessions.

Also removes a doc comment describing msgID and its disambiguating
counter, deleted in 0814f7b, which had come to sit above
messageIDNamespace describing the opposite of the current design; and
corrects the Telegram comment still claiming a redelivered update
"updates the stored turn", the behaviour the immutability fix removed.

Refs archie-core-la1
2026-08-04 22:25:20 +10:00
sam 0fd3398cf8 fix(gateway): stop /compress destroying a message that arrives midway
applyCompress reads the history, computes a summary, then replaces it.
ReplaceMessages removed everything outside the replacement set, so a
chat turn saved in that window was deleted without ever having been
read, let alone summarised. Confirmed on both backends. It is
unrecoverable loss, and it sits outside the write-then-delete guarantee
because the window is in the caller, not the store.

ReplaceMessages now takes the messages the caller is superseding instead
of inferring them from the complement of the replacement. A caller
computes a replacement from a history it read; anything that arrives
afterwards is not part of that, and is now left alone. IDs in both lists
are still kept, so read-modify-write callers are unaffected.

The test fake's ReplaceMessages was a slice assignment, which could not
observe ordering or identity at all -- it now mirrors the real stores.

Refs archie-core-6e0
2026-08-04 22:21:15 +10:00
sam 44d43ef286 fix(gateway): claim the session slot before writing it, not after
resolve saved a newly created session to the store and only then asked
the cache who had won a concurrent race. The loser had already persisted
a session that nothing pointed at: unreachable, but still listed by
/topic and /sessions as "(untitled)", and -- now that resolution goes by
recency -- resurrectable on a later restart. Confirmed by forcing the
interleaving: the store held 2 sessions where it should hold 1.

The claim now happens under the same double-check, ahead of the store
write, so only the winner writes. A failed Save releases the claim, or
the cache would point every later message in that chat at a session that
was never stored.

Refs archie-core-x35
2026-08-04 22:17:39 +10:00
sam a3b9eded5a fix(gateway): stop /topic off poisoning the session cache
handleTopic "off" copied the thread's active session onto the flat key
via setActive(getActive(...)). With nothing cached for that thread it
copied the empty string, and resolve treats map presence as
authoritative -- so it returned "" from then on, permanently, since
cacheActive also returned the empty existing value. Every later message
in that chat persisted under session "", history was shared with any
other poisoned chat, and the turn-lane key became "" so /stop in one
chat cancelled a turn in another.

Reachable as a first message on the webhook and email channels, which
call Route directly without resolving a session first.

setActive now rejects an empty session ID, so no caller can poison the
map whatever it computes, and resolve and cacheActive treat an empty
cached value as a miss so an existing entry repairs itself.

The get-then-set was also two critical sections, silently clobbering a
concurrent /resume or /branch with the value read before the switch.
flattenTopic does it in one.

Refs archie-core-jex
2026-08-04 22:15:28 +10:00
sam d5e016ec59 fix(gateway): resolve the most recently active session, not an arbitrary one
resolve's comment claimed to find "the most recent session matching this
thread and bot" and took whichever the store returned first. On the
NellDB store that is document order -- by session ID -- so it was
effectively arbitrary. The tracker cache is in-memory, so every restart
re-resolves from the store, and the operator could be dropped back into
a session they had left with /new, answering against pre-/new history.
Probed over 10 restarts, 4 landed in the older session. LastActiveAt was
populated on every record and never read.

resolve now picks by LastActiveAt, which also makes it independent of
the two backends disagreeing: SQLite's GetByChannel orders created_at
DESC and NellDB's did not, and the conformance suite did not pin it.

listAllLocked reversed document order and passed that off as recency,
so List -- documented newest-first, and what /sessions and the dashboard
render -- returned reverse-lexicographic order. It now sorts on
LastActiveAt, falling back to CreatedAt for records never touched.

Refs archie-core-7s3
2026-08-04 22:13:55 +10:00
sam dbacb96994 fix(gateway): stop /compress destroying history it should have kept
Two defects, both confirmed on the NellDB and SQLite backends.

IDENTITY. applyCompress rebuilt the protected head and tail as bare
Message{From, Text}, so every message that survived a compression was
deleted and re-inserted with a fresh canonical MessageID, no SourceID
and a timestamp of now. A rebuilt message is a different message: after
one compression, redelivery deduplication was broken for the rest of the
session's life, so a Telegram backlog replay after a restart appended
every surviving message a second time -- 24 became 25 on a redelivery
that was a correct no-op beforehand. That made the dedup guarantee
survive only until the first /compress, and violated
messaging-and-work-intake.md:88-90 and 112-114.

CompressHistory now reports which input messages it protected and the
write-back carries those originals through untouched, so only the
summary is a new record. The summary is attributed to the bot rather
than falling through to "user": CompressHistory emits it with role
"system", which has no representation in the store, so the model was
replaying "[Earlier conversation has been summarised...]" as if the
human had typed it.

DISPATCH. Three ways to rewrite a history the operator did not ask to
rewrite, or to fail to rewrite one they did:

- Bare /compress took the preview branch, and the preview closes with
  "Run /compress to apply" -- the documented way to compress could never
  compress, and the failure-safe path was unreachable.
- /compress focus <topic> discarded the topic and ran a full default
  compression. Nothing implements focused summarisation; it now says so
  instead of doing something else irreversibly.
- /compress here (no count) and any typo fell through to a full
  compression. Both now print usage.

compressPreview also read a fixed 200-message window while reporting the
real count, so the two numbers in one sentence described different
conversations and it advised against compressions that were needed.

Refs archie-core-cz8, archie-core-8k8, archie-core-le6
2026-08-04 22:11:53 +10:00
sam 0814f7b9b4 fix(gateway): give messages durable identity and a SQLite store
Three defects shared one root cause -- messages had no identity of their
own, so the store could not tell a redelivery from a new message:

- SaveMessages destroyed history. Writing a batch replaced records that
  a concurrent write had already stored, and earlier messages vanished.
- A redelivered channel message overwrote the stored one, contradicting
  the append-only sequence in docs/architecture/messaging-and-work-intake
  .md:112-114.
- /compress stranded whatever exceeded one page: deleting 1200 reported
  1000 deleted and left 200 behind.

Messages now carry an application-generated MessageID, derived from
(session, source id) so a redelivery lands on the same record, and
stored records are immutable -- a repeat write is a no-op rather than an
overwrite. SourceID keeps the channel-native id for correlation without
being load-bearing. _ts is written as Unix milliseconds and clamped
monotonic per session: Telegram dates are whole seconds, so replies
arriving inside the same second would otherwise order arbitrarily.
Deletes page with a cursor instead of assuming one page holds everything.

Adds a SQLite session store behind the same interface, per
docs/architecture/messaging-and-work-intake.md:107. A shared conformance
suite runs all 19 groups against both implementations, so the two cannot
drift while the NellDB path is still the one wired up.
2026-08-04 01:44:26 +10:00
sam a66da97ab7 gateway: implement session lifecycle and context commands (archie-core-alm.5)
Add gateway/runtime-owned commands that remain usable when LLM credits
are exhausted:

- /start — acknowledgement without consuming a reply
- /new [name] (alias: /reset) — create a fresh session
- /topic [off|help|<session-id>] — session switching for threaded chats
- /retry — remove last assistant message, replay last user turn
- /undo [N] — remove last N messages from history (default: 1)
- /title [name] — set or display session title
- /branch [name] (alias: /fork) — create child session with inherited history
- /compress [here N|focus topic|--preview|--dry-run] (alias: /compact)

Design:
- sessionTracker maps channel+thread to active session, auto-creating
  implicit sessions with the legacy sessionKey format for backward
  compatibility with existing persisted data
- /new and /branch create explicit UUID-backed sessions
- SessionStore extended with DeleteRecentMessages, MessageCount, and
  SaveMessages for undo/retry/branch/compress operations
- SessionContext extended with Title, ParentSessionID, BranchName
- All commands report 'not configured' when session store isn't wired

Session resolution is wired into the LLM responder so /new, /branch,
and /topic switches affect subsequent chat turns.
2026-07-29 03:43:34 +10:00