LearnNewsExamplesServices
Frontmatter
id13895
title[msg-ingestion] Drain crash-recovery replay + idempotency + retention + observability
stateClosed
labels
enhancementaiarchitecturemodel-experience
assignees[]
createdAtJun 23, 2026, 1:17 AM
updatedAtJun 23, 2026, 1:22 AM
githubUrlhttps://github.com/neomjs/neo/issues/13895
authorneo-opus-grace
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[x] 13893 [msg-ingestion] Durable message WAL + drain daemon (local + cloud parity)
blocking[]
closedAtJun 23, 2026, 1:22 AM

[msg-ingestion] Drain crash-recovery replay + idempotency + retention + observability

Closed v13.1.0/archive-v13-1-0-chunk-5 enhancementaiarchitecturemodel-experience
neo-opus-grace
neo-opus-grace commented on Jun 23, 2026, 1:17 AM

Parent: Epic #13889. The robustness layer that makes the durability invariant hold under failure.

Problem / scope

A durable WAL is only as good as its drain's failure-handling. The daemon must survive crashes mid-drain (replay un-drained entries), not double-ingest (idempotency), bound the WAL (retention past the drain watermark), and be observable — a silent stalled drain is exactly what hid de1d9f0b.

Acceptance Criteria

  • Replay: on restart, the daemon re-drains WAL entries not yet confirmed-persisted (a crash mid-drain loses nothing).
  • Idempotency: re-ingesting a WAL entry is a no-op (the message node is upserted by stable id, never duplicated).
  • Retention: the message WAL is pruned past the confirmed-drain watermark (no unbounded growth), mirroring the memory-wal reconciliation/prune.
  • Observability: drain lag / pending-count / failures are observable (a recordTaskOutcome-style trace or healthcheck field) — so a stalled drain is loud, not silent.

Contract Ledger

  • Adds: replay + watermark + retention + drain observability on the message-ingestion daemon.
  • Reuses: the embed-daemon reconciliation/prune patterns (pruneReconciledWalSegments).

Sequencing

Blocked-by the WAL + drain daemon sub (extends the daemon). Can land after the add_message flip.