Every 1m poll logs issue queued for the same open issue (sam/archie-core#8) again.
Every attempt logs event sink insert failed: err="sdk: revision conflict" (cmd/archied/main.go:373, the event-sink goroutine calling st.InsertEvent).
The daemon's own task consumer logs nats enqueue failed: err="sdk: revision conflict" (internal/daemon/daemon.go:599, calling Store.EnqueueIssue) and NAKs; JetStream redelivers to max-deliver 3 (triples of errors every ~2 min in the log).
The task doc is never persisted, so the enqueue dedup check always passes and the poll re-queues forever. Work never reaches the agent; dashboard shows a stuck/no-op task.
Exact cause
Commit 44f914c bumped github.com/samcharles93/NellDB v0.2.5 -> v0.3.0 touching only go.mod/go.sum. v0.3.0 changed the DocDB rev-cache rebuild:
v0.2.5 sdk/docdb.go reindex() indexed EVERY record by rec.ID, including Archie's meta:task_id_counter / meta:event_id_counter docs.
v0.3.0 sdk/docdb.go reindex() skips any id where isInternalID() is true (sdk/replicate.go:841: id[:5] == "meta:"), intended only for the SDK's own meta:clock/meta:vector bookkeeping but prefix-matching Archie's counter docs too.
v0.3.0 putIn (sdk/docdb.go:208-216) keeps the same conflict rule as v0.2.5: incomingRev != "" && !exists -> sdk.ErrConflict.
Archie's Adapter.nextCounter (internal/nell/adapter.go:957) is a Get -> mutate -> Put RMW that supplies the _rev returned by Get (joinDoc copies the payload, which putIn re-stamps with _rev). After any restart the counter docs are present in the log but absent from the rev cache, so exists=false and every counter Put conflicts. The counter doc can never be written again, so the cache is never repopulated — the failure is permanent for the lifetime of the DB, every process run, from the first poll.
Repro (standalone, module cache, v0.3.0): open log, Put meta:task_id_counter, close, reopen, Get (rev e.g. "1-69bfa51d..."), Put with that rev -> "sdk: revision conflict". First run of a fresh DB succeeds (empty-rev init Put is allowed); every subsequent process restart fails.
Why tests are green: OpenMemory/fresh-log tests initialize the counter via empty-rev Put within the same process; TestPersistenceAcrossRestart reopens a real log but only reads (TaskByIssue) and never exercises the counter RMW after reopen. The production path (restart an existing log, then enqueue) is untested.
Evidence
go.mod: github.com/samcharles93/NellDB v0.3.0 (no replace).
Module cache diff !nell!d!b@v0.2.5 vs @v0.3.0: sdk/docdb.go reindex()/putIn(), sdk/replicate.go isInternalID.
/tmp/nell-repro: deterministic ErrConflict on counter RMW after reopen.
Fix direction
Options, in order of preference: (1) pin back to v0.2.5 (verified-2026-07-28 semantics) until adapter is migrated; (2) stop supplying _rev in nextCounter (drop rev from the RMW doc before Put — v0.3.0 continues the chain from the cached rev when incomingRev == ""), but that breaks the MVCC chain for the counter when the cache lacks it; (3) upstream: isInternalID should match exact SDK ids (meta:clock, meta:vector), not the meta: prefix, or reindex should index meta docs. Any fix must add a red test: open persisted log, EnqueueIssue, reopen, EnqueueIssue again -> no conflict.
Imported from beads archie-core-1786637491729-74-ee80f814 · GitHub #95 · type bug · priority P2 · created 2026-08-08 · closed 2026-08-08
## Symptom (observed 2026-08-05, task dev session)
- Every 1m poll logs `issue queued` for the same open issue (sam/archie-core#8) again.
- Every attempt logs `event sink insert failed: err="sdk: revision conflict"` (cmd/archied/main.go:373, the event-sink goroutine calling st.InsertEvent).
- The daemon's own task consumer logs `nats enqueue failed: err="sdk: revision conflict"` (internal/daemon/daemon.go:599, calling Store.EnqueueIssue) and NAKs; JetStream redelivers to max-deliver 3 (triples of errors every ~2 min in the log).
- The task doc is never persisted, so the enqueue dedup check always passes and the poll re-queues forever. Work never reaches the agent; dashboard shows a stuck/no-op task.
## Exact cause
Commit 44f914c bumped github.com/samcharles93/NellDB v0.2.5 -> v0.3.0 touching only go.mod/go.sum. v0.3.0 changed the DocDB rev-cache rebuild:
- v0.2.5 sdk/docdb.go reindex() indexed EVERY record by rec.ID, including Archie's meta:task_id_counter / meta:event_id_counter docs.
- v0.3.0 sdk/docdb.go reindex() skips any id where isInternalID() is true (sdk/replicate.go:841: id[:5] == "meta:"), intended only for the SDK's own meta:clock/meta:vector bookkeeping but prefix-matching Archie's counter docs too.
- v0.3.0 putIn (sdk/docdb.go:208-216) keeps the same conflict rule as v0.2.5: incomingRev != "" && !exists -> sdk.ErrConflict.
Archie's Adapter.nextCounter (internal/nell/adapter.go:957) is a Get -> mutate -> Put RMW that supplies the _rev returned by Get (joinDoc copies the payload, which putIn re-stamps with _rev). After any restart the counter docs are present in the log but absent from the rev cache, so exists=false and every counter Put conflicts. The counter doc can never be written again, so the cache is never repopulated — the failure is permanent for the lifetime of the DB, every process run, from the first poll.
Repro (standalone, module cache, v0.3.0): open log, Put meta:task_id_counter, close, reopen, Get (rev e.g. "1-69bfa51d..."), Put with that rev -> "sdk: revision conflict". First run of a fresh DB succeeds (empty-rev init Put is allowed); every subsequent process restart fails.
Why tests are green: OpenMemory/fresh-log tests initialize the counter via empty-rev Put within the same process; TestPersistenceAcrossRestart reopens a real log but only reads (TaskByIssue) and never exercises the counter RMW after reopen. The production path (restart an existing log, then enqueue) is untested.
## Evidence
- go.mod: github.com/samcharles93/NellDB v0.3.0 (no replace).
- git show 44f914c --stat: go.mod, go.sum, .gitignore only; internal/nell/ untouched since.
- Module cache diff !nell!d!b@v0.2.5 vs @v0.3.0: sdk/docdb.go reindex()/putIn(), sdk/replicate.go isInternalID.
- /tmp/nell-repro: deterministic ErrConflict on counter RMW after reopen.
## Fix direction
Options, in order of preference: (1) pin back to v0.2.5 (verified-2026-07-28 semantics) until adapter is migrated; (2) stop supplying _rev in nextCounter (drop rev from the RMW doc before Put — v0.3.0 continues the chain from the cached rev when incomingRev == ""), but that breaks the MVCC chain for the counter when the cache lacks it; (3) upstream: isInternalID should match exact SDK ids (meta:clock, meta:vector), not the meta: prefix, or reindex should index meta docs. Any fix must add a red test: open persisted log, EnqueueIssue, reopen, EnqueueIssue again -> no conflict.
---
_Imported from beads `archie-core-1786637491729-74-ee80f814` · GitHub #95 · type bug · priority P2 · created 2026-08-08 · closed 2026-08-08_
sam
added the mediumbug labels 2026-10-27 13:36:59 +00:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Symptom (observed 2026-08-05, task dev session)
issue queuedfor the same open issue (sam/archie-core#8) again.event sink insert failed: err="sdk: revision conflict"(cmd/archied/main.go:373, the event-sink goroutine calling st.InsertEvent).nats enqueue failed: err="sdk: revision conflict"(internal/daemon/daemon.go:599, calling Store.EnqueueIssue) and NAKs; JetStream redelivers to max-deliver 3 (triples of errors every ~2 min in the log).Exact cause
Commit 44f914c bumped github.com/samcharles93/NellDB v0.2.5 -> v0.3.0 touching only go.mod/go.sum. v0.3.0 changed the DocDB rev-cache rebuild:
Archie's Adapter.nextCounter (internal/nell/adapter.go:957) is a Get -> mutate -> Put RMW that supplies the _rev returned by Get (joinDoc copies the payload, which putIn re-stamps with _rev). After any restart the counter docs are present in the log but absent from the rev cache, so exists=false and every counter Put conflicts. The counter doc can never be written again, so the cache is never repopulated — the failure is permanent for the lifetime of the DB, every process run, from the first poll.
Repro (standalone, module cache, v0.3.0): open log, Put meta:task_id_counter, close, reopen, Get (rev e.g. "1-69bfa51d..."), Put with that rev -> "sdk: revision conflict". First run of a fresh DB succeeds (empty-rev init Put is allowed); every subsequent process restart fails.
Why tests are green: OpenMemory/fresh-log tests initialize the counter via empty-rev Put within the same process; TestPersistenceAcrossRestart reopens a real log but only reads (TaskByIssue) and never exercises the counter RMW after reopen. The production path (restart an existing log, then enqueue) is untested.
Evidence
Fix direction
Options, in order of preference: (1) pin back to v0.2.5 (verified-2026-07-28 semantics) until adapter is migrated; (2) stop supplying _rev in nextCounter (drop rev from the RMW doc before Put — v0.3.0 continues the chain from the cached rev when incomingRev == ""), but that breaks the MVCC chain for the counter when the cache lacks it; (3) upstream: isInternalID should match exact SDK ids (meta:clock, meta:vector), not the meta: prefix, or reindex should index meta docs. Any fix must add a red test: open persisted log, EnqueueIssue, reopen, EnqueueIssue again -> no conflict.
Imported from beads
archie-core-1786637491729-74-ee80f814· GitHub #95 · type bug · priority P2 · created 2026-08-08 · closed 2026-08-08