LearnNewsExamplesServices
Frontmatter
id15802
titleLocal in-process WAL drains in mc-server (memory + message)
stateClosed
labels
enhancementaiarchitecture
assigneesneo-opus-ada
createdAtJul 24, 2026, 1:16 PM
updatedAtJul 24, 2026, 9:53 PM
githubUrlhttps://github.com/neomjs/neo/issues/15802
authorneo-fable
commentsCount1
parentIssue15798
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[x] 15799 Plane identity: opaque planeId as a paired AiConfig leaf + pure-defaults twin
blocking[x] 15806 Parity pilot posture: cloned-snapshot plane, write disposition, measured baselines
closedAtJul 24, 2026, 9:53 PM

Local in-process WAL drains in mc-server (memory + message)

Closed Backlog/active-chunk-9 enhancementaiarchitecture
neo-fable
neo-fable commented on Jul 24, 2026, 1:16 PM

Context

Phase-2 leaf of epic #15798 (Discussion #15595). The cloud profile already runs in-process WAL drains inside mc-server (NEO_MEMORY_WAL_IN_PROCESS_DRAIN); locally, dedicated drain daemons do the work per seat — N seats can each spawn drainers against the one symlinked WAL, coordinated only by the cross-clone drain lock. Parity means one drain topology in both worlds: a single in-container/in-process drainer per plane, retiring that contention surface locally.

The Problem

The local drain topology is the alternate-reality WAL class's home turf: the memory-wal leaf is a shared-plane member (divergence #8), the historical allowlist scar orphaned records for ~8 days precisely here, and this week's production fixes (#15748-class: drains dark while writes "succeed") were the cloud face of the same coin. Two drain paths = two realities to verify.

The Architectural Reality

  • The cloud drain exists and is production-proven (the enable flag + drain machinery; drainLock.mjs handles cross-clone contention today).
  • Plane identity (#15799) gives the drainer its subject: one drainer per declared plane, not per seat/clone.
  • Drain observability feeds the continuity-receipt vocabulary (D#15758 cross-consumption): drained-clean disposition is a receipt field the pilot (phase 5) and the rollout epic both consume.
  • 2026-07-24 intake truth-fold (steward-ratified, Ada's V-B-A): both in-process drain paths are ALREADY built, wired, and lock-protected (ai/mcp/server/memory-core/Server.mjs — memory drain via acquireDrainLock, message drain via acquireMessageDrainLock; both daemons acquire the counterpart locks; drainLock modules carry unit specs). The lock design makes concurrent draining UNREPRESENTABLE: a second LIVE host refuses and fails loud, naming the holder; a dead holder's lock is reclaimed via a process.kill(pid, 0) liveness probe. The genuinely new work is the drain-disposition receipt (AC3) + the baseline measurement path (AC5).

The Fix (one PR)

Verify + surface the in-process drain path (memory + message WALs) for the local parity profile: plane-scoped single-drainer semantics; drainLock refusal semantics proven (a second drain host is unconstructable-by-refusal, never reconciled after the fact); drain-disposition surfaced (drained-clean flag + counts) on the existing observability path.

Acceptance Criteria

  • [re-homed → #15805] Parity profile drains memory + message WALs in-process; no dedicated local drain daemon required for a parity seat. (Depends on #15805's parity-profile switch — blocked_by #15803 + #15801; not deliverable from this leaf alone. Verification rides #15805 against this leaf's shipped receipts — this open box does NOT block #15802's close; annotated per the PR #15834 review's close-target audit, 2026-07-24T18:22Z.)
  • Sole-drainer refusal proven (restated 2026-07-24, supersedes the original "exactly-once race" wording): a second drain host against the same plane REFUSES to start, fails loud, and names the current holder; a dead holder's lock is reclaimed via the liveness probe so a crash never wedges the drain. Prevention, not reconciliation — there is no race to make exactly-once, because concurrent draining is unrepresentable by construction.
  • Drain disposition (drained-clean, counts) readable post-drain — the receipt field the pilot baseline consumes. (PR #15834: the four-state receipt reads a producer-declared outstanding residue, never pre-drain pending; boundary-crossing integration test proves it.)
  • Non-parity (stdio) seats keep the existing daemon path untouched. (PR #15834: the wiring is additive — a record call + an exposed getter per loop; the 32 existing daemon specs pass unchanged.)
  • memory-wal baseline measurement path documented (feeds the phase-5 fork-then-replay vs dual-journal falsifier). (PR #15834: group by metadata.agentIdentity with an explicit unattributed bucket; dated corpus counts live in the PR evidence, not durable JSDoc.)

Delivery split (steward-ratified 2026-07-24): this leaf delivers AC2 (refusal witness) + AC3 + AC4 + AC5 now; AC1 completes on #15805's profile and is verified there against this leaf's receipts.

Contract Ledger

Consumed surfaces changed by PR #15834 (the drain-disposition receipt). Full diff-level ledger lives in the PR body; this is the authority-surface copy Emmy's Contract Completeness gate requires on the ticket.

Surface Source of authority Behavior Fallback / edge
drainWalOnce() summary +outstanding this leaf; the receipt is the sole consumer pending − embedded − compensated (post-cycle residue) additive field; pending stays a pre-drain observation and must never be read as work-left
drainMessageWalOnce() summary +outstanding this leaf observed − drained additive; inactive path returns outstanding: 0
startDrainLoop() / startMessageDrainLoop(){stop, getDisposition} this leaf; the phase-5 pilot consumer getDisposition(){state, drainedClean, reason, counts, at} both JSDocs corrected from {stop}-only; options.now documented
createDrainDispositionTracker().recordCycle(summary) this leaf reads summary.outstanding; !Number.isFiniteunobserved never grades clean from a summary that omits the residue field

No AiConfig / ADR-0019 surface touched; no MCP tool or OpenAPI description changed.

Out of Scope

Retiring the drain daemons repo-wide (post-cutover); the pilot's write-disposition contract (phase-5 leaf); any embed-daemon changes.

Related

Epic #15798 · blocked_by #15799 · D#15595 rationale 1 + divergence #8 · #15748-class production anchors · borrowed-authority caveat as in #15799.

Origin Session ID: 758f110e-a429-4597-beb0-5fd3ec7d1bfc Retrieval Hint: query_raw_memories("in-process WAL drain parity single drainer per plane drainLock")

tobiu referenced in commit 4329d92 - "feat(ai): the drain receipt — four states, so cleanliness cannot be an absence (#15802) (#15834) on Jul 24, 2026, 9:53 PM
tobiu closed this issue on Jul 24, 2026, 9:53 PM