feat(memory): builtin engine implementing the domain/memory contract

Part of archie-core-1786637499480 (ship the file-writing store as the
default memory engine).

Records the missing design decision first (migration-decisions.md#5,
"authoritative memory record and revision model" was explicitly open):
internal/memory/builtin.Store addresses blocks by content substring with
no stable per-block ID, which the new Record{ID}/Forget(id) contract
needs. Decided and implemented: an ID assigned once at Write via a
leading `<!--mem:<uuid>-->` marker line, stripped back out of
Record.Content on read, located by that marker (not content) for
Forget -- so an unrelated edit can never orphan a Forget. Identity maps
to a directory named hex(sha256(identity)), not the identity string, so
an arbitrary identity can never traverse outside the configured root.
Kind maps to a builtin section name via the store's own
ValidateSectionName, so an invalid Kind fails the same way an invalid
section already does.

internal/infrastructure/memory.BuiltinEngine wraps one builtin.Store per
identity and implements domain/memory.MemoryEngine end to end
(Write/Query/List/Forget, Lifecycle, Manifest). Lives under
internal/infrastructure per organisation.md ("infrastructure implements
domain contracts"), not internal/domain/memory, which stays
contract-only.

internal/memory/builtin is untouched -- confirmed via git status and its
own test suite passing unmodified, per this issue's acceptance
criterion. This engine is not yet wired into daemon bootstrap or the
config surface (archie-core-1786637490043); it proves the contract
holds for the builtin store the same way curator's reference
implementations prove the curator contract, and registers/runs cleanly
through domain/memory.Registry (TestBuiltinEngineRegistersAndRuns
ThroughTheRegistry).
This commit is contained in:
sam committed 2026-09-02 02:36:10 +10:00
1 parent ec86e8c5c1
commit 7b39f7bd0c
4 files changed
+580

No files matched your search

+41
View File
@@ -222,6 +222,47 @@ and tests must decide:
Conversation branches do not create another memory scope. A memory action has
the same scoped effect regardless of which Conversation originated it.
**`internal/domain/memory` ownership — DECIDED (2026-09-02).** The typed
engine family lives at `internal/domain/memory`
(archie-core-1786637490069-11-e245c170), mirroring `internal/domain/curator`'s
contract/registrar/registry split. Not owned within the Agent domain: memory
is consumed by more than agent turns (curators, per archie-core-1786637499636)
and the plugin engine rule requires its own family, not a capability bolted
onto another domain's contract.
**Authoritative record and revision model for the builtin engine — DECIDED
(2026-09-02),** scoped to unblock archie-core-1786637499480 (ship the
file-writing store as the default engine). This decides only the builtin
engine's mapping onto the new `Record{ID}`/`Forget(id)` contract, not the
memory model in general or a future non-file engine's model:
- `internal/memory/builtin.Store` is unchanged. It has no concept of a
stable per-block ID — blocks are addressed by content substring — and nothing
in this decision requires changing that; existing `internal/memory` tests
keep passing unmodified, per that issue's acceptance criteria.
- A new infrastructure adapter (`internal/infrastructure/memory`, per
`organisation.md`'s "infrastructure implements domain contracts" rule — not
under `internal/domain/memory`, which stays contract-only) wraps one
`builtin.Store` per identity.
- **Identity → directory:** `hex(sha256(identity))`, not the identity string
itself. Closes the path-traversal question without an allowlist regex, and
sidesteps collisions from identities that differ only in characters a
filesystem treats as equivalent.
- **Record ID:** assigned once at `Write`, independent of content, so a later
edit elsewhere in the file can never orphan a `Forget`. Realized as a
leading marker line (`<!--mem:<ulid>-->`) on the block `builtin.Store.Add`
writes; `Query`/`List` strip it back out of `Record.Content` before
returning, and `Forget(id)` locates the block by that unique marker via the
store's existing substring-based `Remove`. The marker is a pragmatic,
reversible choice — plain-text records stay human-readable and
human-editable, at the cost of one marker line an operator editing the file
by hand should leave alone. Revisit if/when provenance representation
(still open, below) settles on something richer.
- **Kind → section:** `Observation.Kind` (default `"general"` when empty) is
the section name, reusing `builtin.ValidateSectionName` — so an
invalid `Kind` fails `Write` with the same validation error `memory_edit`
already returns for an invalid section today, not a new error shape.
### 6. Runtime and process boundaries — DECIDED
`archied` is the native resident orchestration and interactive-chat process.
@@ -0,0 +1,229 @@
// Package memory implements domain/memory.MemoryEngine against concrete
// backends. See docs/architecture/migration-decisions.md#5-memory-placement-and-storage
// for the record/identity model this package's builtin engine follows.
package memory
import (
"context"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"sync"
"github.com/google/uuid"
domainmemory "github.com/samcharles93/archie-core/internal/domain/memory"
"github.com/samcharles93/archie-core/internal/memory/builtin"
)
// EngineName is the name BuiltinEngine registers under and the default the
// config surface (archie-core-1786637490043) resolves to when no engine is
// configured.
const EngineName = "builtin"
// markerPrefix opens the leading line every block written through this
// engine carries, embedding the record's stable ID. See the migration
// decision this package implements: an ID assigned once at Write, never
// derived from content, so a later edit elsewhere in the file can never
// orphan a Forget.
const (
markerPrefix = "<!--mem:"
markerSuffix = "-->"
)
// defaultSectionName is the section an Observation with an empty Kind is
// written to.
const defaultSectionName = "general"
// BuiltinEngine implements domain/memory.MemoryEngine over one
// internal/memory/builtin.Store per identity. internal/memory/builtin is
// unchanged by this package -- see the migration decision this
// implements for why.
type BuiltinEngine struct {
root string
maxFileBytes int
mu sync.Mutex
stores map[string]*builtin.Store // keyed by identity, lazily created
}
// NewBuiltinEngine builds an engine that persists each identity's
// observations under its own subdirectory of root, named
// hex(sha256(identity)) -- never the identity string itself, so an
// arbitrary identity can never traverse outside root and two identities
// differing only in filesystem-equivalent characters can never collide.
// maxFileBytes <= 0 uses the store's own default.
func NewBuiltinEngine(root string, maxFileBytes int) *BuiltinEngine {
return &BuiltinEngine{root: root, maxFileBytes: maxFileBytes, stores: make(map[string]*builtin.Store)}
}
func (e *BuiltinEngine) Name() string { return EngineName }
func (e *BuiltinEngine) Version() string { return "1" }
func (e *BuiltinEngine) Manifest() domainmemory.Manifest {
return domainmemory.Manifest{RequiresNetwork: false}
}
func (e *BuiltinEngine) Bind(domainmemory.Registrar) {}
func (e *BuiltinEngine) Start(context.Context) error { return nil }
func (e *BuiltinEngine) Health(context.Context) domainmemory.Health {
return domainmemory.Health{Status: domainmemory.HealthHealthy}
}
func (e *BuiltinEngine) Stop(context.Context) error { return nil }
// storeFor returns the identity's store, creating and caching it on first
// use.
func (e *BuiltinEngine) storeFor(identity string) (*builtin.Store, error) {
e.mu.Lock()
defer e.mu.Unlock()
if s, ok := e.stores[identity]; ok {
return s, nil
}
dir := filepath.Join(e.root, identityDir(identity))
if err := os.MkdirAll(dir, 0o755); err != nil {
return nil, fmt.Errorf("memory: builtin engine: identity directory %s unusable: %w", dir, err)
}
s, err := builtin.NewStore(filepath.Join(dir, "OBSERVATIONS.md"), e.maxFileBytes)
if err != nil {
return nil, err
}
e.stores[identity] = s
return s, nil
}
// identityDir maps an identity to a filesystem-safe, collision-resistant
// directory name.
func identityDir(identity string) string {
sum := sha256.Sum256([]byte(identity))
return hex.EncodeToString(sum[:])
}
// sectionFor returns the section an Observation's Kind maps to, and
// validates it the same way builtin already validates any section name --
// so an invalid Kind fails with the same error shape memory_edit already
// returns for an invalid section, not a new one.
func sectionFor(kind string) (string, error) {
if kind == "" {
kind = defaultSectionName
}
if err := builtin.ValidateSectionName(kind); err != nil {
return "", err
}
return kind, nil
}
func (e *BuiltinEngine) Write(_ context.Context, obs domainmemory.Observation) (domainmemory.Record, error) {
if err := obs.Validate(); err != nil {
return domainmemory.Record{}, err
}
section, err := sectionFor(obs.Kind)
if err != nil {
return domainmemory.Record{}, err
}
s, err := e.storeFor(obs.Identity)
if err != nil {
return domainmemory.Record{}, err
}
id := uuid.NewString()
marked := markerPrefix + id + markerSuffix + "\n" + obs.Content
if _, err := s.Add(section, marked); err != nil {
return domainmemory.Record{}, err
}
return domainmemory.Record{
ID: id,
Identity: obs.Identity,
Kind: section,
Content: obs.Content,
At: obs.At,
Metadata: obs.Metadata,
}, nil
}
func (e *BuiltinEngine) Query(ctx context.Context, q domainmemory.Query) ([]domainmemory.Record, error) {
if err := q.Validate(); err != nil {
return nil, err
}
all, err := e.List(ctx, q.Identity)
if err != nil {
return nil, err
}
if q.Text == "" {
return limitRecords(all, q.Limit), nil
}
var matched []domainmemory.Record
for _, r := range all {
if strings.Contains(r.Content, q.Text) {
matched = append(matched, r)
}
}
return limitRecords(matched, q.Limit), nil
}
func limitRecords(records []domainmemory.Record, limit int) []domainmemory.Record {
if limit > 0 && len(records) > limit {
return records[:limit]
}
return records
}
func (e *BuiltinEngine) List(_ context.Context, identity string) ([]domainmemory.Record, error) {
if identity == "" {
return nil, errors.New("memory: builtin engine: identity must not be empty")
}
s, err := e.storeFor(identity)
if err != nil {
return nil, err
}
return parseRecords(identity, s.Render()), nil
}
func (e *BuiltinEngine) Forget(_ context.Context, id string) error {
if id == "" {
return errors.New("memory: builtin engine: id must not be empty")
}
marker := markerPrefix + id + markerSuffix
e.mu.Lock()
stores := make([]*builtin.Store, 0, len(e.stores))
for _, s := range e.stores {
stores = append(stores, s)
}
e.mu.Unlock()
// Forgetting an id that does not exist is not an error (contract.go's
// Forget doc): the end state -- id absent -- already holds whether or
// not any store ever had it, so checking every known identity's store
// and returning nil regardless of a miss is correct, not a fallback
// masking a real failure.
for _, s := range stores {
section, ok := sectionOf(s.Render(), id)
if !ok {
continue
}
if _, err := s.Remove(section, marker, true); err != nil {
return err
}
return nil
}
return nil
}
// sectionOf finds which section of rendered carries a block whose marker
// names id, so Forget can call Store.Remove with the section it actually
// requires without a mutating probe. ok is false if no block does.
func sectionOf(rendered, id string) (section string, ok bool) {
for _, rec := range parseRecords("", rendered) {
if rec.ID == id {
return rec.Kind, true
}
}
return "", false
}
@@ -0,0 +1,240 @@
package memory
import (
"context"
"os"
"path/filepath"
"testing"
domainmemory "github.com/samcharles93/archie-core/internal/domain/memory"
)
func newTestEngine(t *testing.T) *BuiltinEngine {
t.Helper()
return NewBuiltinEngine(t.TempDir(), 0)
}
func TestBuiltinEngineWriteQueryListForgetRoundTrip(t *testing.T) {
t.Parallel()
ctx := context.Background()
e := newTestEngine(t)
rec, err := e.Write(ctx, domainmemory.Observation{Identity: "u1", Kind: "preference", Content: "likes tabs over spaces"})
if err != nil {
t.Fatalf("Write() = %v, want nil", err)
}
if rec.ID == "" {
t.Fatal("Write() returned an empty ID")
}
if rec.Content != "likes tabs over spaces" {
t.Fatalf("Write() content = %q, want the observation content back", rec.Content)
}
list, err := e.List(ctx, "u1")
if err != nil {
t.Fatalf("List() = %v, want nil", err)
}
if len(list) != 1 || list[0].ID != rec.ID {
t.Fatalf("List() = %#v, want the one written record", list)
}
found, err := e.Query(ctx, domainmemory.Query{Identity: "u1", Text: "tabs"})
if err != nil {
t.Fatalf("Query() = %v, want nil", err)
}
if len(found) != 1 || found[0].ID != rec.ID {
t.Fatalf("Query(tabs) = %#v, want the matching record", found)
}
miss, err := e.Query(ctx, domainmemory.Query{Identity: "u1", Text: "nonexistent"})
if err != nil {
t.Fatalf("Query() = %v, want nil", err)
}
if len(miss) != 0 {
t.Fatalf("Query(nonexistent) = %#v, want none", miss)
}
if err := e.Forget(ctx, rec.ID); err != nil {
t.Fatalf("Forget() = %v, want nil", err)
}
after, err := e.List(ctx, "u1")
if err != nil {
t.Fatalf("List() after Forget = %v, want nil", err)
}
if len(after) != 0 {
t.Fatalf("List() after Forget = %#v, want empty", after)
}
}
func TestBuiltinEngineForgetUnknownIDIsNotAnError(t *testing.T) {
t.Parallel()
e := newTestEngine(t)
if err := e.Forget(context.Background(), "does-not-exist"); err != nil {
t.Fatalf("Forget(unknown) = %v, want nil: the end state (id absent) already holds", err)
}
}
func TestBuiltinEngineIsolatesIdentities(t *testing.T) {
t.Parallel()
ctx := context.Background()
e := newTestEngine(t)
if _, err := e.Write(ctx, domainmemory.Observation{Identity: "u1", Content: "u1's secret"}); err != nil {
t.Fatalf("Write(u1) = %v, want nil", err)
}
if _, err := e.Write(ctx, domainmemory.Observation{Identity: "u2", Content: "u2's secret"}); err != nil {
t.Fatalf("Write(u2) = %v, want nil", err)
}
u1, err := e.List(ctx, "u1")
if err != nil {
t.Fatalf("List(u1) = %v, want nil", err)
}
if len(u1) != 1 || u1[0].Content != "u1's secret" {
t.Fatalf("List(u1) = %#v, want only u1's record", u1)
}
u2, err := e.List(ctx, "u2")
if err != nil {
t.Fatalf("List(u2) = %v, want nil", err)
}
if len(u2) != 1 || u2[0].Content != "u2's secret" {
t.Fatalf("List(u2) = %#v, want only u2's record", u2)
}
}
func TestBuiltinEngineIdentityDoesNotTraverseOutsideRoot(t *testing.T) {
t.Parallel()
root := t.TempDir()
e := NewBuiltinEngine(root, 0)
if _, err := e.Write(context.Background(), domainmemory.Observation{
Identity: "../../../etc/passwd",
Content: "should stay contained",
}); err != nil {
t.Fatalf("Write() = %v, want nil", err)
}
entries, err := os.ReadDir(root)
if err != nil {
t.Fatalf("ReadDir(root) = %v", err)
}
if len(entries) != 1 {
t.Fatalf("root has %d entries, want exactly 1 hashed identity directory: %v", len(entries), entries)
}
if entries[0].Name() == "../../../etc/passwd" || filepath.IsAbs(entries[0].Name()) {
t.Fatalf("identity directory = %q, want a hashed name, not the raw identity", entries[0].Name())
}
}
func TestBuiltinEngineWriteRejectsInvalidKindLikeAnInvalidSection(t *testing.T) {
t.Parallel()
e := newTestEngine(t)
_, err := e.Write(context.Background(), domainmemory.Observation{Identity: "u1", Kind: "not/a/valid/section!", Content: "x"})
if err == nil {
t.Fatal("Write(invalid kind) = nil, want an error matching builtin's section validation")
}
}
func TestBuiltinEngineWriteDefaultsEmptyKindToGeneral(t *testing.T) {
t.Parallel()
ctx := context.Background()
e := newTestEngine(t)
rec, err := e.Write(ctx, domainmemory.Observation{Identity: "u1", Content: "no kind given"})
if err != nil {
t.Fatalf("Write() = %v, want nil", err)
}
if rec.Kind != defaultSectionName {
t.Fatalf("Kind = %q, want %q", rec.Kind, defaultSectionName)
}
}
func TestBuiltinEngineRejectsInvalidObservationAndQuery(t *testing.T) {
t.Parallel()
ctx := context.Background()
e := newTestEngine(t)
if _, err := e.Write(ctx, domainmemory.Observation{Content: "no identity"}); err == nil {
t.Fatal("Write(no identity) = nil, want an error")
}
if _, err := e.Query(ctx, domainmemory.Query{Text: "no identity"}); err == nil {
t.Fatal("Query(no identity) = nil, want an error")
}
if _, err := e.List(ctx, ""); err == nil {
t.Fatal("List(\"\") = nil, want an error")
}
if err := e.Forget(ctx, ""); err == nil {
t.Fatal("Forget(\"\") = nil, want an error")
}
}
func TestBuiltinEngineRecordSurvivesUnrelatedEdits(t *testing.T) {
// The migration decision this implements: an ID assigned at Write,
// independent of content, so writing and forgetting unrelated records
// never orphans another record's Forget.
t.Parallel()
ctx := context.Background()
e := newTestEngine(t)
first, err := e.Write(ctx, domainmemory.Observation{Identity: "u1", Content: "first"})
if err != nil {
t.Fatalf("Write(first) = %v, want nil", err)
}
second, err := e.Write(ctx, domainmemory.Observation{Identity: "u1", Content: "second"})
if err != nil {
t.Fatalf("Write(second) = %v, want nil", err)
}
if err := e.Forget(ctx, second.ID); err != nil {
t.Fatalf("Forget(second) = %v, want nil", err)
}
list, err := e.List(ctx, "u1")
if err != nil {
t.Fatalf("List() = %v, want nil", err)
}
if len(list) != 1 || list[0].ID != first.ID || list[0].Content != "first" {
t.Fatalf("List() after forgetting second = %#v, want only first intact", list)
}
}
func TestBuiltinEngineLifecycleAndManifest(t *testing.T) {
t.Parallel()
e := newTestEngine(t)
if got := e.Name(); got != EngineName {
t.Errorf("Name() = %q, want %q", got, EngineName)
}
if e.Manifest().RequiresNetwork {
t.Error("Manifest().RequiresNetwork = true, want false for a local file store")
}
if err := e.Start(context.Background()); err != nil {
t.Errorf("Start() = %v, want nil", err)
}
if got := e.Health(context.Background()); got.Status != domainmemory.HealthHealthy {
t.Errorf("Health() = %v, want healthy", got)
}
if err := e.Stop(context.Background()); err != nil {
t.Errorf("Stop() = %v, want nil", err)
}
}
func TestBuiltinEngineRegistersAndRunsThroughTheRegistry(t *testing.T) {
// Acceptance criterion: the builtin store implements the engine
// contract and registers as the default.
t.Parallel()
r := domainmemory.NewRegistry(domainmemory.Registrar{})
e := newTestEngine(t)
if err := r.Register(e); err != nil {
t.Fatalf("Register() = %v, want nil", err)
}
if got, ok := r.Get(EngineName); !ok || got != domainmemory.MemoryEngine(e) {
t.Fatalf("Get(%q) = (%v, %v), want the registered builtin engine", EngineName, got, ok)
}
if err := r.Start(context.Background()); err != nil {
t.Fatalf("registry Start() = %v, want nil", err)
}
if err := r.Stop(context.Background()); err != nil {
t.Fatalf("registry Stop() = %v, want nil", err)
}
}
+70
View File
@@ -0,0 +1,70 @@
package memory
import (
"strings"
domainmemory "github.com/samcharles93/archie-core/internal/domain/memory"
)
// sectionHeaderPrefix matches internal/memory/builtin's own format (its
// sectionHeaderPrefix constant is unexported): a "## " line starts a named
// section, and blocks within it are separated by one blank line. This
// package only ever reads what it wrote through builtin.Store.Add, so it
// only needs to read that one documented format back, not the general
// case builtin.Store itself parses.
const sectionHeaderPrefix = "## "
// parseRecords reads every marked block out of rendered and returns it as
// a Record scoped to identity. A block with no marker was not written by
// this engine (e.g. hand-edited) and is skipped: this engine's contract is
// only ever asked about records it assigned an ID to.
func parseRecords(identity, rendered string) []domainmemory.Record {
var records []domainmemory.Record
section := ""
var blockLines []string
flush := func() {
if len(blockLines) == 0 {
return
}
if rec, ok := parseBlock(identity, section, strings.Join(blockLines, "\n")); ok {
records = append(records, rec)
}
blockLines = nil
}
for line := range strings.SplitSeq(rendered, "\n") {
if name, ok := strings.CutPrefix(line, sectionHeaderPrefix); ok {
flush()
section = strings.TrimSpace(name)
continue
}
if strings.TrimSpace(line) == "" {
flush()
continue
}
blockLines = append(blockLines, line)
}
flush()
return records
}
// parseBlock splits one block back into its marker-carried ID and content.
// ok is false for a block with no recognizable marker.
func parseBlock(identity, section, block string) (domainmemory.Record, bool) {
first, rest, _ := strings.Cut(block, "\n")
id, ok := strings.CutPrefix(first, markerPrefix)
if !ok {
return domainmemory.Record{}, false
}
id, ok = strings.CutSuffix(id, markerSuffix)
if !ok {
return domainmemory.Record{}, false
}
return domainmemory.Record{
ID: id,
Identity: identity,
Kind: section,
Content: rest,
}, true
}