Embed drain: boot guard mechanically enforces the sole-drainer invariant

Context

Born from @neo-opus-ada's Approve+Follow-Up review on PR #12859 (review PRR_kwDODSospM8AAAABCoKPBg, Depth-Floor challenge): the WAL drain now has two host modes — the orchestrator-supervised embed daemon (local profile) and the in-server loop under memoryWal.inProcessDrain (containerized/single-process deployments). The sole-drainer invariant that the purge-race compensation depends on ("a marker I didn't write = a purge tombstone") is config-DECLARED (JSDoc + leaf comment + mutually-exclusive default profiles) but not mechanically ENFORCED.

Live latest-open sweep: checked the latest 20 open issues at 2026-06-10T20:2xZ — no equivalent (closest: #12854, ada's primary-dev-sync migrate-config automation — adjacent config-freshness surface, distinct concern). A2A claim sweep (last-step, per the #12856 gate): list_messages({status:'all', limit:15}) — no competing [lane-claim]/[lane-intent] in the herd window; the reviewer explicitly routed this filing to the PR author.

Release classification: post-v13 — deliberately NOT attached to Project 12. This is Approve+Follow-Up-born hardening that cannot fire on the default deployment profiles (the verdict itself certifies the PR ships without it). Filed per the scope-gate principle the swarm is codifying tonight (operator friction→gold item; sibling substrate ticket links back here as its first applied instance).

The Problem

A misconfiguration that enables BOTH drain hosts on one WAL directory (e.g. embedDaemonEnabled resolving true on a deployment whose MC server also sets NEO_MEMORY_WAL_IN_PROCESS_DRAIN=true) silently breaks marker semantics: each loop interprets the other's embed markers as purge tombstones mid-flight, turning the compensation logic against legitimately-embedded records — spurious collection.deletes of healthy memories. Integrity-class failure, silent, config-distance from the happy path.

The Architectural Reality

• ai/daemons/embed/drainCycle.mjs — startDrainLoop (shared host) + drainWalOnce (the compensation that assumes sole-drainer); the invariant is documented at both.
• ai/daemons/embed/daemon.mjs — enforceSingleton (PID lock) protects daemon-vs-daemon only; it cannot see an in-server loop.
• ai/mcp/server/memory-core/Server.mjs boot() — the in-process host, gated by the memoryWal.inProcessDrain leaf.
• The two enable-flags live in DIFFERENT config trees (Tier-1 orchestrator.localOnly.embedDaemonEnabled vs memory-core memoryWal.inProcessDrain), so a single enum leaf cannot trivially replace them without cross-tree coupling.

The Fix (recommended shape — implementer may improve)

A drain-lock file per WAL directory, claimed by whichever host starts first:

1. acquireDrainLock({dir, owner}) in memoryWalStore.mjs (or a sibling helper): atomic wx write of <dir>/.drain-lock containing {pid, owner: 'daemon'|'in-process', startedAt}; stale-lock detection via process.kill(pid, 0) liveness probe (dead pid → reclaim).
2. Both hosts acquire before startDrainLoop; on conflict with a LIVE holder: refuse + fail loud naming the other holder (no takeover — the daemon's existing PID-takeover stays scoped to daemon-vs-daemon succession).
3. Release on clean shutdown (daemon cleanup handler; server stop path); stale locks self-heal via the liveness probe.
4. Falsifiers: lock acquired/released; second host refuses while first alive; stale lock reclaimed; daemon-succession takeover still works.

Avoided Traps

• ❌ Single drainMode enum leaf — the two flags live in different config trees (Tier-1 vs memory-core); an enum forces cross-tree coupling or a breaking flag migration. Revisit post-v13 if config unification lands.
• ❌ Reusing the daemon's PID-takeover against the server — the daemon SIGTERMs competing PID holders; pointed at the MC server process that would kill the server. Refusal, not takeover.
• ❌ Runtime marker-provenance tagging (writer-id in each marker line) — heavier wire-format change; the lock prevents the conflict instead of detecting it after corruption.

Acceptance Criteria

• AC1: a drain-lock primitive guards startDrainLoop callers; second-host start against a live holder refuses with an actionable error naming the holder.
• AC2: stale locks (dead pid) self-heal; daemon-vs-daemon succession (existing takeover) unaffected.
• AC3: clean shutdown releases the lock (daemon signal path; server stop path).
• AC4: falsifier specs for acquire/refuse/stale-reclaim/release in the drainCycle or store spec.

Out of Scope

• Config-tree unification of the two enable flags (post-v13 architecture item).
• Any change to drain semantics themselves.

Related

• PR #12859 (origin review thread) / #12840 (the drain substrate).
• #12854 (adjacent config-freshness automation, distinct).

Origin Session ID: 00ab373d-0219-4093-948b-f9d30ecd4c7b

Retrieval Hint: "sole-drainer invariant drain-lock mutual exclusion embed daemon inProcessDrain"