LearnNewsExamplesServices
Frontmatter
id13893
title[msg-ingestion] Durable message WAL + drain daemon (local + cloud parity)
stateClosed
labels
enhancementaiarchitecturemodel-experience
assignees[]
createdAtJun 23, 2026, 1:17 AM
updatedAtJun 23, 2026, 1:21 AM
githubUrlhttps://github.com/neomjs/neo/issues/13893
authorneo-opus-grace
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[x] 13895 [msg-ingestion] Drain crash-recovery replay + idempotency + retention + observability, [x] 13894 [msg-ingestion] Switch add_message to WAL-first + remove the synchronous model-hit (durability invariant)
closedAtJun 23, 2026, 1:21 AM

[msg-ingestion] Durable message WAL + drain daemon (local + cloud parity)

Closed v13.1.0/archive-v13-1-0-chunk-5 enhancementaiarchitecturemodel-experience
neo-opus-grace
neo-opus-grace commented on Jun 23, 2026, 1:17 AM

Parent: Epic #13889 (daemon-based A2A message ingestion). The core durable pipe.

Problem / scope

A2A messages must persist durably without a synchronous model-hit. Mirror the memory ingestion pattern: a durable append-only message WAL + a daemon that drains it → graph-persist + embed, load-aware. This is the pipe the add_message flip (sibling sub) writes into.

Acceptance Criteria

  • V-B-A first: confirm the current add_message persist mechanism (write-through to SQLite vs in-memory buffer) and where the durability gap is — empirical anchor: de1d9f0b (created, add_message returned an id, but absent from Nodes + GraphLog + WAL). Shape the WAL boundary from that finding.
  • Durable append-only message WAL (mirror memory-wal): one durable record per add_message, written before the request returns.
  • Drain daemon (mirror ai/daemons/embed/): drains the message WAL → graph-persist (Nodes + SENT_BY/SENT_TO/DELIVERED_TO edges; the GraphLog insert-trigger fires) → embed. Load-aware — yields to heavy maintenance (REM / KB-sync), like the embed daemon.
  • Local ↔ cloud parity (operator constraint, load-bearing): the daemon runs + reaches the WAL / graph / embedding-model in BOTH the local native topology AND the cloud multi-container topology, mirroring the memory embed-daemon's placement + wiring (compose service, volumes, reachability). Without this, messages persist in one reality but not the other.

Contract Ledger

  • Adds: the message WAL (pattern: memory-wal) + the message-ingestion daemon (pattern: ai/daemons/embed/).
  • Reuses: the graph persist path (GraphService.upsertNode / linkNodes), the embed pipeline.

Sequencing

First sub of #13889 — the pipe. The add_message-flip + replay/robustness subs build on it.