LearnNewsExamplesServices
Frontmatter
id14797
titleMailboxService read-state resurrection — repair re-projects `readAt:null` from the WAL over committed mark_reads on every list/get
stateClosed
labels
bugaiarchitecture
assigneesneo-opus-ada
createdAt4:32 PM
updatedAt5:53 PM
githubUrlhttps://github.com/neomjs/neo/issues/14797
authorneo-opus-ada
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAt5:53 PM

MailboxService read-state resurrection — repair re-projects readAt:null from the WAL over committed mark_reads on every list/get

Closed Backlog/active-chunk-4 bugaiarchitecture
neo-opus-ada
neo-opus-ada commented on 4:32 PM

Context — the mark-eaten / read-state-rollback mechanism, now located in code

Split from #14426 (which closed on the node-LOSS grade) per @neo-fable-clio's re-triage recommendation: the distinct defect here is write REVERSION on an existing node's edge state — committed mark_read writes resurrected to readAt: null. Ten datums across #14426 + #14576 (Clio's forensics + @neo-fable's amplitude series 31→55→101→262 + both-ends specimens) converged on one mechanism, which I have verified against the code in ai/services/memory-core/MailboxService.mjs.

This is critical-class: it breaks the standing "mailbox read-status is the truth" invariant that lets agents dismiss at-least-once wake redeliveries. Every wake currently delivers inflated digests; committed reads silently revert; V-B-A loops that re-fetch prior coordination are poisoned.

The mechanism (code-verified — exact sites)

  1. The WAL is pure intake — readAt: null forever. Messages enter the message WAL with creation properties; mark_read mutates the GRAPH projection (setDeliveryEdgeReadAt :721 for broadcast edges / DM node property write :1676), never the WAL record. Correct by design — the WAL is intake, not state.
  2. _projectMessageWalRecord (:1068) re-materializes from the WAL, hardcoding readAt: null:
    • Broadcast (:1108–1114): re-links the DELIVERED_TO edge with a readAt: null literal (:1110) — wiping the committed per-recipient read-state.
    • DM (:1095–1100): upsertNode({…, properties: messageProperties}) overwrites the node with the WAL's original properties (readAt: null).
  3. repairMessageGraphIntegrity (:1195) calls it on every read. listMessages invokes repair unconditionally (:1287); getMessage at :1461. It fires for any record whose getMessageGraphProjectionIssues (:528) is non-empty, gated by hasMailboxGraphProjectionGap() — a stale in-memory-cache-over-SQLite predicate (the file's own :1292 comment: peer-process writes are invisible without vicinity re-hydration). A cold/stale cache perceives an intact projection as a gap → repairs → resurrects. The mailbox eats its own read-state as a side effect of being read.

Why it explains every symptom family

  • Eaten mark_reads (both-ends specimen): mark_read commits to the projection; the next listMessages runs the :1287 repair → re-projects readAt: null. Reverter and victim in one transcript.
  • Oscillating unread counts (267→13→144 in 19 min): per-call cache staleness flips the gap predicate; each bounded repair batch resurrects a different window.
  • Restart amplitude growth (31→55→101→262): cold cache post-restart = maximal perceived gap = largest sweep = most resurrections; grows with WAL size.
  • Phantom wake replays (#14576 datums 5–10): resurrected-unread real messages re-entering the wake digest window — the digest pipeline is honest; its input lies.

Fix shape (falsifier-first, one PR)

  1. Repair merges mutable post-delivery state. Before re-projecting, read the CURRENT edge/node readAt + archivedAt (+ any retraction marker) and carry them onto the re-projected structure. The WAL stays pure intake; repair becomes structure-only. Distinguish the drain path (genuine first projection — readAt: null correct) from the repair path (re-projection over live state — must preserve).
  2. Gap predicate must not trust a stale cache. Re-hydrate the relevant vicinity (getAdjacentNodes — the codebase's own documented primitive, :1292) before evaluating hasMailboxGraphProjectionGap().

Acceptance Criteria

  • Falsifier-first specs land RED on current code, GREEN after the fix:
    • Mark a broadcast DELIVERED_TO edge read → repairMessageGraphIntegrity({ids:[messageId]}) → assert readAt survives re-projection.
    • DM/node carrier variant (node-property readAt survives).
    • Unread-count stability across two consecutive listMessages calls with a cold cache.
  • _projectMessageWalRecord, on the repair path, preserves current readAt/archivedAt on both carriers (broadcast edge + DM node); the WAL record stays the structural source, never the read-state source.
  • hasMailboxGraphProjectionGap() re-hydrates vicinity before deciding, so an intact projection is never perceived as a gap by a stale cache.
  • No listMessages/getMessage call can revert a committed mark_read (the invariant, asserted).
  • #14576's phantom-wake-replay symptoms verified as downstream (quiesce once the falsifier passes).

Out of scope

  • The original node-LOSS grade (#14426) — a distinct destructive-sync-rewrite class.
  • Tiered wake policy (#14576) — the consumer; this fixes its poisoned input, not the policy.

Forensics: @neo-fable-clio (mechanism + datum corpus), @neo-fable (amplitude series). Code-verification (exact line sites, _projectMessageWalRecord overwrite + repair-on-every-read call chain): @neo-opus-ada. Refs #14426 (closed — node-loss grade) · #14576 (downstream wake corpus).

tobiu closed this issue on 5:53 PM
tobiu referenced in commit 67dda7f - "fix(ai): mailbox repair goes surgical — read-state survives re-projection (#14806) on 5:53 PM