Task and event persistence now uses a dedicated SQLite sidecar derived from the configured state prefix, matching the independently persisted conversation store. The configured path now means a state-path prefix rather than a database file so fresh SQLite state never attempts to interpret the previous storage format. Remove the retired implementation and dependency completely, strengthen SQLite migrations and concurrency behavior, restore backend-neutral coverage, and align configuration, operational guidance, UI labels, and deployment documentation with the new storage and host-daemon topology.
14 KiB
Messaging and Work Intake
Status: Current state researched; foundation decisions in progress
Date: 2026-07-28
Tracking issue: #73
Purpose
Archie is a personal assistant and general agentic system. Its interactions include conversation, questions, notes, reminders, goals, message handling, troubleshooting, infrastructure operations, server management, source changes, and open-ended future capabilities.
A message to an agent is therefore not inherently a work request, and invoking an agent is not inherently a WorkflowExecution.
Messaging and work intake are separate from the Agent System's durable workflow execution model:
- messaging preserves interactions with people and external systems;
- an agent turn interprets and responds to an interaction using its available capabilities;
- work intake handles the optional promotion of an interaction into durable, workflow-backed work;
- the Agent System owns Workflow definitions and creates WorkflowExecutions.
Forge issues, Jira issues, Telegram messages, emails, webhooks, and future channels remain peer external message or artifact sources. None becomes an execution model merely because it can lead to work.
Confirmed boundary
Work intake MUST NOT create a WorkflowExecution directly.
Inbound interactions normally route to the Agent selected by the channel binding. The selected Agent retains its own memory, user knowledge, conversation continuity, specialisation, and capability context. This continuity is independent of the underlying model or provider.
Each selected Agent acts through its own immutable IdentityID. Interaction and resulting action records preserve both that acting identity and the initiating user identity when one exists.
Agent memory is private by default. Interaction context MAY additionally use user-wide memory and intentionally global shared memory, but another Agent's agent-wide memory is not part of the selected Agent's context.
Multiple users may interact with the same Agent. The channel binding and inbound message MUST resolve both AgentID and initiating User IdentityID before conversation or memory context is loaded. Sharing an Agent MUST NOT implicitly share one user's private memory with another user.
An interaction may assemble memory from four scopes: global shared, the selected Agent's agent-wide memory, the initiating user's authorized user-wide memory, and the private Agent-user relationship memory addressed by both IDs.
Conversation identity
Every conversation is identified by the complete tuple:
AgentID
+ UserIdentityID
+ ChannelBindingID
+ ExternalConversationID
+ ThreadID
ThreadID has an explicit empty value for channels without threads. No element
of this key may be replaced by a display name, configured bot username, channel
type, or other mutable label.
Agent and user memory continue across conversations according to their memory scope. Immediate conversation history and delivery correlation remain isolated to the identified channel conversation or thread. Moving between Telegram, email, or another channel does not create a new Agent or user, but it does create a distinct conversation unless an explicit linking operation relates them.
Conversation and message events support projections for channel-entry metrics, use cases, volumes, outcomes, latency, capability usage, and other product or operational analysis. Metrics are derived read models; they MUST NOT become mutable counters embedded in the conversation aggregate or change message routing semantics.
Canonical messages
Every inbound, assistant, tool-call, and tool-result message has an immutable application-generated MessageID. Platform identifiers are external correlation, not substitutes for the canonical ID.
Messages are addressable because an interaction is more than text. A message may contain assistant content, reasoning data subject to its retention policy, one or more tool calls, a tool result with authoritative execution metadata, attachments, reply relationships, edits, or other typed content. Stable MessageIDs also allow sessions to branch from an exact prior point.
The canonical session and message implementation must supply:
- application-generated stable message IDs;
- user, assistant, and tool roles;
- assistant tool calls and correlated tool-result messages;
- authoritative tool-result status, timing, error, truncation, and size metadata;
- session lifecycle and active-request state;
- session-to-agent-instance and parent-session lineage;
- durable SQLite round trips and timestamp validation.
These semantics must use the confirmed Agent, user, channel-binding, and conversation ownership model.
The canonical message store MUST preserve messages as immutable records and represent later edits, deletions, and branches through related records or events.
Archie requires exact branch-point correlation using stable MessageIDs.
Conversation branches
A branch is a continuation of a previous conversation at an exact message. A child conversation records:
ParentConversationID
ForkMessageID
Messages remain an append-only linear sequence within each conversation. The
child's effective initial context is the parent's history through
ForkMessageID; messages after that point in the parent are not inherited.
After the fork, parent and child are isolated:
- child messages, tool calls, tool results, capability actions, and errors do not appear in the parent conversation;
- later parent messages do not silently enter the child;
- returning to the parent resumes its unchanged history;
- the child may itself be branched using the same rules;
- branch lineage remains queryable for navigation, audit, and metrics.
This supports building substantial context about a subject and then performing one or more contextual subtasks without polluting the original conversation. For example, a user may branch from a project discussion, create Jira issues in the child using inherited context, and then return to the project discussion without adding the Jira tool transcript to the parent.
Branching does not copy mutable message rows. History is resolved through the immutable parent conversation and fork MessageID, followed by the child's own message sequence.
Branch isolation applies to conversation history, not to Agent learning or memory. A memory action performed in a branch has exactly the same semantics as the same action performed in any other conversation. It writes to its selected global, agent-wide, user-wide, or Agent-user relationship scope.
Memory provenance MAY retain the source ConversationID and MessageIDs for explanation and audit. Retrieval and effect are determined by memory scope and access rules, not by the conversation from which the memory originated. Returning to a parent does not insert the child's tool transcript into the parent history, but scoped memories learned in the child remain available to future retrieval wherever that scope is visible.
Changing sessions or branching therefore controls immediate conversational context without preventing the Agent from learning durable knowledge.
Target ownership
The Messaging domain lives at internal/domain/messaging. It owns:
- canonical Messages and typed message content;
- Conversations, branches, and lineage;
- channel bindings and external source correlation;
- replies, acknowledgements, and outbound delivery attempts;
- message and conversation lifecycle events;
- domain-defined repository and delivery service contracts;
- projections required for conversation navigation and metrics.
Work intake is a separate cohesive domain at internal/domain/workintake. It
owns admission, validation, deterministic routing, and accepted work requests
for the optional transition into durable workflow-backed execution.
Telegram, email, webhook, forge, Jira, and future messaging systems are infrastructure adapters implementing Messaging contracts. They are users of the domain; they do not own Messages, Conversations, work admission, or WorkflowExecution semantics.
The Agent decides whether an ordinary interaction is answered directly, uses a capability, changes or retrieves Agent-owned information, or proposes durable workflow-backed work.
Explicitly configured automation MAY enter work intake without a conversational Agent turn when the external trigger and admission rule already unambiguously request durable work. This avoids spending an LLM turn to rediscover a deterministic routing decision.
When an interaction requires durable workflow-backed execution:
- a channel adapter produces a channel-neutral inbound message;
- the acting agent handles or interprets the interaction;
- work intake validates, admits, and routes a proposed work request;
- work intake emits an accepted work request with its source correlation, acting identity, requested outcome, inputs, and routing evidence;
- the Agent System consumes that request, selects a specific Workflow version, and creates the WorkflowExecution.
The accepted work request is a cross-domain handoff, not an alternate execution aggregate. The Agent System is the sole authority that creates a valid WorkflowExecution.
An interaction MAY instead be completed directly by an agent turn, delegated to a tool or capability, recorded as a note or reminder, answered conversationally, or retained for later context. Those paths MUST NOT require a Workflow or WorkflowExecution.
A Workflow MAY add reusable, versioned structure to any of those behaviours. Scheduled personal-assistant routines, message triage, reminders, infrastructure operations, and other capability sequences are valid Workflow definitions. The distinction is whether durable structured execution is useful, not whether the subject originated from a forge.
Canonical execution language
- Workflow: reusable, versioned behaviour definition.
- WorkflowExecution: durable execution of a specific Workflow version.
- WorkflowStep: a defined operation in a Workflow.
- StepExecution: the recorded execution and outcome of a WorkflowStep.
- Attempt: a retry within the same WorkflowExecution.
External issues, messages, requests, goals, and notes are not synonyms for these concepts.
Current implementation
Current intake is fragmented:
- Telegram, email, and webhook adapters translate payloads into
gateway.Message, while an unused richerMessageEventcontract exists alongside it. gateway.Routercombines local channel commands, conversational LLM routing, and direct task lifecycle operations.- Telegram is the only channel with production LLM and durable conversation wiring. Email and webhook free text currently have no LLM responder.
- Forge polling bypasses the gateway and directly enqueues store rows or NATS task messages.
- Chat
/spawndirectly creates astore.Taskand invents a synthetic forge issue number to satisfy the existing persistence key. - Forge replies and chat approval commands implement separate waiting-human paths.
- Message delivery, acknowledgement, retry, deduplication, and source correlation are platform-specific side effects rather than one durable model.
store.Taskcollapses an external source reference and durable execution state.
Current hazards
- Treating every agent interaction as a work request would remove Archie's personal-assistant role and force conversation and immediate capability use through an inappropriate workflow lifecycle.
- Treating every accepted work request as a WorkflowExecution would let intake or channel code select versions and construct invalid execution state outside the Agent System.
- Source identity is incomplete: sessions use configured bot usernames, tasks use identity strings, and external message IDs are generally absent.
- SQLite and NATS deduplication are based on forge coordinates rather than a channel-neutral source-message reference.
- Replies and acknowledgements have no durable delivery-attempt lifecycle, so partial success and retry behaviour differ by adapter.
- Production code has two competing channel-neutral message contracts.
- The existing Archie session implementation lacks the required tool-call semantics and exact branch lineage and MUST be brought to the canonical contract.
Decisions still required
- the cohesive owner and final location of messaging behaviour;
- the cohesive owner and final location of work-intake admission and routing;
- the canonical inbound message, conversation, and source-correlation model;
- the boundary between an agent turn, direct capability use, and a proposed durable work request;
- the exact relationship between an Agent, its acting IdentityID, its user, and its channel bindings;
- outbound response, acknowledgement, delivery-attempt, retry, deduplication, and failure semantics;
- the smallest accepted-work-request contract consumed by the Agent System;
- migration of forge polling, replies, chat commands, email, webhook, Telegram, and Jira integrations.
Non-goal: semantic search over chat messages
Semantic/vector search over chat messages is an explicit non-goal until usage data shows a real need.
Message search is lexical only: session-scoped SQLite FTS5 queries paged via
MessageQuery{Query,Limit,Offset} → MessagePage. Do not add embeddings
configuration or vector columns to message records.
The rationale is cost without a demonstrated use case. Measured 2026-08-03 with
embeddinggemma:300m — note this was never an infrastructure problem, the model
was already pulled and ai-sdk/provider/ollama already has embed.go:
- 121 ms median warm per message on the write path, 12.5 s cold;
- 768 dims = 3072 bytes per vector, roughly 15× a typical message payload;
- chronological reads would carry vector data they do not use on the hot path of every turn;
- similarity search requires an explicit indexing and retention design.
If this is revisited, vectors belong in a sibling table so chronological reads never load them.