Replay message WAL into mailbox graph idempotently

Context

Parent epic: #13889

The parent epic needs three separate deliverables to avoid bundling: the WAL-first add_message acceptance boundary (#13891), the local/cloud drain topology (#13890), and this replay leaf. This ticket owns the derived graph projection step: accepted message WAL records must become MESSAGE graph nodes plus mailbox delivery edges idempotently, and failures must leave retryable pending work instead of losing the acknowledged message.

Release classification: boardless - data-integrity hardening under an active epic, not currently attached to a release board.

Live latest-open sweep: checked the latest 20 open issues immediately before creation on 2026-06-22T23:14Z; no equivalent child ticket found. Scope-specific live sweep for message WAL + replay + graph returned only parent epic #13889 and sibling leaves #13890 / #13891. A2A in-flight sweep checked the latest 30 messages and found no competing lane-claim for this child scope.

The Problem

Once add_message becomes WAL-first, the graph write becomes derived work. That derived work still has to preserve mailbox semantics: direct DMs need SENT_BY and SENT_TO, broadcasts need DELIVERED_TO receipts, optional provenance/thread/ticket/concept edges should be recreated consistently, and replay must be safe after restart or partial projection.

The current implementation performs all of this in-request. A replay path that is not idempotent would either duplicate delivery edges on retry or mark work as complete before delivery-critical edges exist. Either failure reintroduces the parent bug in another form: acknowledged messages become missing, duplicated, or unroutable.

The Architectural Reality

ai/services/memory-core/MailboxService.mjs:640 writes the MESSAGE node inline, then ai/services/memory-core/MailboxService.mjs:651 through ai/services/memory-core/MailboxService.mjs:663 create the delivery-critical SENT_BY, SENT_TO, and broadcast DELIVERED_TO edges. ai/services/memory-core/MailboxService.mjs:665 through ai/services/memory-core/MailboxService.mjs:673 add optional provenance/thread/ticket/concept edges.

The memory WAL precedent already separates acceptance from graph projection. ai/services/memory-core/MemoryService.mjs:585 schedules bounded retry, ai/services/memory-core/MemoryService.mjs:629 projects one accepted WAL record into graph, and ai/services/memory-core/MemoryService.mjs:711 drains graph-pending WAL records as a restart/backstop path.

ai/services/memory-core/helpers/memoryWalStore.mjs:185 provides graph projection markers, while ai/services/memory-core/helpers/memoryWalStore.mjs:271 reads pending records using separate marker streams. Message replay may need its own store/marker schema, but it should preserve the same durable marker principle: pending until projection is complete.

The Fix

Implement a message WAL replay/projection path that consumes accepted message WAL records and creates mailbox graph state idempotently:

• Reconstruct the exact MESSAGE node properties and delivery-critical edges from the accepted WAL record.
• Treat delivery-critical graph state as all-or-retry: no projection marker until node plus required routing edges are present.
• Make replay idempotent across process restart, repeated drain attempts, and partial graph success.
• Preserve optional edges (ORIGINATES_IN, IN_REPLY_TO, PART_OF_THREAD, RELATED_SESSION, REFERENCES_TICKET, TAGGED_CONCEPT) without letting optional cull/failure block delivery-critical completion.
• Surface projection failures in structured logs/summary counts without losing pending records.
• Keep model/vector/search population out of this leaf unless it is strictly required to complete mailbox graph projection; otherwise state the deferral to #10150 or a later drain leaf.

If implementation introduces a new .mjs module instead of extending existing service seams, run structural pre-flight at pickup time before selecting the directory.

Contract Ledger Matrix

Decision Record impact

Aligned with the existing memory WAL graph-projection substrate. No ADR change expected for this child.

Decision Record: Not needed.

Acceptance Criteria

• A pending message WAL record can be replayed into a MESSAGE graph node with the original stable id.
• Direct-message replay creates delivery-critical SENT_BY and SENT_TO edges before marking the record projected.
• Broadcast replay creates required per-recipient DELIVERED_TO receipts before marking the record projected.
• Replaying the same WAL record multiple times is idempotent and does not duplicate delivery semantics.
• A partial projection failure leaves the record pending and retryable; no projected marker is written until delivery-critical graph state is complete.
• Optional provenance/thread/ticket/concept edges are recreated when possible but do not block delivery-critical projection.
• Drain/replay exposes summary counts or logs sufficient to diagnose stuck records.
• The PR declares whether message vector/search population is in scope here or deferred to #10150 / later drain work.

Out of Scope

• WAL-first add_message acceptance; that is #13891.
• Local/cloud drain host topology and deployment wiring; that is #13890.
• Full semantic-search UX for related messages; defer or relate to #10150.
• Fade/apoptosis MESSAGE exemption hardening.

Avoided Traps

• Do not mark a record projected after only the node exists; mailbox routing depends on delivery-critical edges.
• Do not let optional-edge failures block a direct/broadcast message from becoming deliverable.
• Do not rely on in-memory GraphService cache state as the source of replay truth; the accepted WAL record is the authority.
• Do not add model-dependent work back onto the add_message request path.

Related

Parent: #13889

Related: #13890, #13891, #10150

Origin Session ID: db5b2ecf-db91-4b7d-9498-ccef00426a1c

Handoff Retrieval Hints: A2A message WAL replay graph idempotent mailbox delivery edges daemon drain; MailboxService.addMessage SENT_BY SENT_TO DELIVERED_TO MemoryService drainPendingGraphProjections