LearnNewsExamplesServices
Frontmatter
id15821
titlemark_read returns a read receipt even when the durable write is skipped (false-ack)
stateClosed
labels
bugaiarchitecture
assigneesneo-opus-grace
createdAtJul 24, 2026, 4:34 PM
updatedAtJul 24, 2026, 10:24 PM
githubUrlhttps://github.com/neomjs/neo/issues/15821
authorneo-opus-grace
commentsCount2
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJul 24, 2026, 10:24 PM

mark_read returns a read receipt even when the durable write is skipped (false-ack)

Closed Backlog/active-chunk-9 bugaiarchitecture
neo-opus-grace
neo-opus-grace commented on Jul 24, 2026, 4:34 PM

Context

Reported by @neo-fable-clio as an empirical friction observation on my #15448 read-state lane: three times on 2026-07-24 her mailbox resurfaced messages as UNREAD that she had verifiably mark_read earlier, each occurrence correlated with an MC server restart/reconnect (her harness saw the disconnect/reconnect notices). Pattern: recent mark_read writes vanish across the restart; older read-state survives.

This is distinct from the restore-path read-state loss fixed under #15448 / PR #15808 (that is --mode replace restore truncating the graph before re-apply). This is the restart/reconnect lifecycle, one lifecycle over — and it has a confirmed code-level defect that the restore fix does not touch.

The Problem

Confirmed defect (this is the premise, and it is real independent of the mechanism below):

setDeliveryEdgeReadAt (ai/services/memory-core/MailboxService.mjs:1125) mutates the in-memory edge unconditionally, then gates the durable write on a condition — but mark_read returns its success receipt regardless of whether that durable write ran:

async function setDeliveryEdgeReadAt(edge, readAt) {
    setRecordProperties(edge, {...getRecordProperties(edge), readAt});   // in-memory — ALWAYS
    const db = GraphService.db;
    if (db?.autoSave && db.storage) {          // durable write — CONDITIONAL
        await db.storage.addEdges([edge]);     // synchronous better-sqlite3 txn — durable WHEN it runs
        db.acknowledgeLocalMutations?.();
    }
}
// caller returns { messageId, readAt, status: 'read' } unconditionally

So whenever db.autoSave is false (or db.storage is absent) at the moment a mark_read executes, the tool returns status: 'read' while nothing durable is written — and a restart drops the in-memory mutation. An acknowledged write that was never persisted is a false receipt. This is the same "confirmation that cannot fail" class the read-state cluster keeps surfacing: the ack asserts a durability the code did not deliver.

Trace Update (same session) — the false-ack is LATENT; the firing premise is corrected

After filing, @neo-fable-clio's reframe ("the ack may lie always and the restart merely reveal it") prompted one more check that redirected the diagnosis, and I am correcting the premise rather than driving a fix on the un-rechecked version:

The mc-server constructs its graph DB as Neo.create(CoreDatabase, {id, storage}) (GraphService.mjs:139) — it does NOT pass autoSave, so autoSave takes its config default of true (Database.mjs:34). In steady state the durable write at :1133 does run, so a normal mark_read persists and the ack is truthful. The false-ack is therefore a LATENT code smell, not a defect shown to fire in the normal path — it requires autoSave === false at mark-time, and every autoSave=false window I found (six sites in Database.mjs, now including :543/:596) is synchronous, so a mark_read cannot execute inside one. Three trigger hypotheses now weakened or falsified: sync-window interleave (falsified), WAL-checkpoint loss (WAL is crash-safe), steady-state autoSave=false (falsified — mc-server default is true).

Leading hypothesis is now the RECONNECT REBUILD, not the write-path gate. Database.syncCache() (Database.mjs:~124) is the delta-sync/reconnect handler: it reads storage.getDeltaLog(lastSyncId), removes delta.invalidEdges from the in-memory cache, and relies on lazy reload from storage (its own comment at :~178 names the reload). If a DELIVERED_TO edge that was marked-read is flagged in invalidEdges and the lazy reload returns a version lacking the readAt (stale delta-log entry, or a readAt that reached the edge object but not the log the delta reads), the mark is lost and it becomes visible exactly at reconnect — which is the reporter's wall-clock correlation. This is the same rebuild-from-a-captured-source family as #15431

4th falsification (same session, static-trace limit reached): syncCache is WEAKENED too, and I am not leaving it standing as "leading" unqualified. Reading syncCache fully (Database.mjs:124-184): it only removes invalidated edges from the in-memory cache and relies on lazy reload from storage — it does not touch storage. In steady state the readAt is durably in storage (autoSave true → the :1133 write ran), so the lazy reload would re-read the committed edge with its readAt and restore it. For syncCache to drop a readAt, the storage version itself would have to lack it — which is the original false-ack (write skipped) or a restore/reseed truncation (#15808 class), not a syncCache defect. So four mechanisms are now weakened or falsified (sync-window interleave · WAL-checkpoint loss · steady-state autoSave=false · syncCache-reload-drops-readAt), and I have hit the limit of what static code reading can resolve here. The mechanism is now blocked on RUNTIME INSTRUMENTATION, not on more of my tracing — the reporter's syncCache/:1133 probe (does the lost edge's storage row carry the readAt at the moment of loss?) is the only thing that discriminates "never persisted" from "persisted then dropped." Static tracing did its job — it eliminated four wrong fixes — but it cannot confirm the cause. and the original #15448 incident, one lifecycle over from the restore path #15808 fixes. Recorded as the leading hypothesis, NOT asserted — I have shown the path exists that could drop a readAt across syncCache, not that it does. The discriminating probe below is updated to target it.

The Architectural Reality

  • MailboxService.setDeliveryEdgeReadAtai/services/memory-core/MailboxService.mjs:1125.
  • Database.autoSave defaults true (ai/graph/Database.mjs:34) but is toggled false at six sites (delta-sync invalid-node/edge pruning, vicinity load, others), each restoring wasAutoSave after a synchronous block.
  • ai/graph/storage/SQLite.mjs:53 runs journal_mode = WAL. addEdges (:394) is a synchronous better-sqlite3 transaction — durable (and crash-safe under WAL) once it runs.

Mechanism — OPEN QUESTION, recorded so it is not re-walked

I traced two candidate mechanisms for why the confirmed defect fires under the reconnect correlation and weakened both; neither is proven, and the ticket does not assert one:

  1. mark_read interleaves with an autoSave = false window. ⚠️ Weakened: the four autoSave = false sites I inspected in Database.mjs are synchronous — no await between toggle-off and toggle-back — so a JS mark_read cannot execute inside them. This mechanism only survives if a longer-lived, async autoSave=false state exists (a bulk import/load that spans awaits) that I have not located.
  2. The durable write runs, but a restart/reseed reads a different or truncated store. ⚠️ Weakened: WAL is crash-safe, so a committed addEdges should survive a plain restart. This survives only under the symlinked-WAL multi-writer topology (#15802) or a concurrent restore/reseed event (which is the #15808 class — and matches the reporter's odd "12:30Z marks resurfacing late at 14:19Z" anomaly, suggesting a second mechanism for that instance).

The cheapest discriminating probe, retargeted after the trace update: log at Database.syncCache when an entry in delta.invalidEdges is a DELIVERED_TO edge that currently carries a readAt, and log the readAt of its reloaded replacement. If the reload drops the readAt, the reconnect-rebuild hypothesis is confirmed and the fix moves to syncCache / the delta log, not the mark_read write path. A MailboxService:1133 autoSave-state log stays useful only to confirm the false-ack remains dormant.

The Fix (direction — the confirmed defect is fixable now; the mechanism informs the rest)

The false-ack shape is wrong regardless of which mechanism triggers it, and is the first thing to fix:

  • Make the receipt conditional on the durable write. setDeliveryEdgeReadAt must not let its caller return status:'read' when the if branch was skipped — either write unconditionally (drop the autoSave gate for read-state, since a receipt is not a bulk-load optimisation target), or return a signal the caller turns into a non-ack / retryable state when the durable path did not run.
  • Then resolve the mechanism (the log-line probe) to decide whether a second fix is needed on the autoSave-window or store-identity side.

Out of Scope

  • The restore-path read-state loss — #15448 / PR #15808 (--mode replace truncate-before-reapply). Distinct lifecycle; already fixed pending merge.
  • WAL drain topology / parity — #15802. Different concern (drainer coordination, not receipt durability).

Avoided Traps

  • Asserting the reconnect-interleave mechanism as proven. I did that in a first A2A to the reporter and retracted it after finding the windows are synchronous — the confirmed defect is the false-ack shape, not a specific interleave, and the ticket premise is scoped to what is proven.
  • Folding into #15808. That PR is merge-pending and scoped to the restore path; a distinct lifecycle bug rides its own ticket.

Acceptance Criteria

  • (hardening — the latent smell) mark_read (via setDeliveryEdgeReadAt) does not return status: 'read' unless the read receipt was durably written. This closes the false-ack shape regardless of whether it is the loss mechanism — a dormant false-ack is still a landmine for any future autoSave=false optimisation.
  • (the actual loss) the reconnect/syncCache path does not drop a committed readAt when invalidating-and-reloading a DELIVERED_TO edge — proven by the retargeted probe, then fixed at whichever of syncCache / delta-log / storage the probe implicates.
  • A regression spec: with the durable path disabled/skipped, mark_read does not report success (or reports a retryable non-ack) — proving the ack tracks durability, red-then-green.
  • The mechanism open-question is resolved or explicitly deferred with the discriminating-probe result attached; if a second (autoSave-window or store-identity) defect is confirmed, it gets its own AC or follow-up.

Related

Related: #15448 Related: #15808 Related: #15802

Reporter: @neo-fable-clio (reconnect-correlation evidence: three 2026-07-24 occurrences; batches at ~13:14Z / ~13:42Z / 12:30Z resurfaced after reconnects at ~13:5xZ / ~14:06Z / ~14:19Z).

Origin Session ID: a4efc85c-aec8-43da-9774-9c735da0b244

Retrieval Hint: query_raw_memories("mark_read false-ack durable write skipped autoSave restart read-state")

tobiu referenced in commit c675736 - "fix(ai): a mailbox receipt must not outrun its durable write (#15821) (#15824) on Jul 24, 2026, 10:24 PM
tobiu closed this issue on Jul 24, 2026, 10:24 PM