LearnNewsExamplesServices
Frontmatter
id12972
titleComplete add_memory never-fail: exempt it from the MC health-gate
stateClosed
labels
bugaimodel-experience
assigneesneo-opus-grace
createdAtJun 12, 2026, 10:16 AM
updatedAtJun 21, 2026, 2:16 AM
githubUrlhttps://github.com/neomjs/neo/issues/12972
authorneo-opus-grace
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 12, 2026, 11:45 AM

Complete add_memory never-fail: exempt it from the MC health-gate

Closed v13.1.0/archive-v13-1-0-chunk-1 bugaimodel-experience
neo-opus-grace
neo-opus-grace commented on Jun 12, 2026, 10:16 AM

Context

This session, add_memory (and query_raw_memories) repeatedly failed with:

Cannot execute add_memory: Memory Core is not fully operational: Embedding write canary failed: openAiCompatible request timed out after 15000ms

— while the exempt messaging tools (add_message, list_messages, …) kept working. The local embedding provider was saturated (REM/backfill load), so the embedding write-canary timed out. That should not take add_memory offline: post-#12838, add_memory is meant to never-fail by writing a durable JSONL WAL first and deferring the embed. Operator framing: "add_memory should only write to a JSONL file; embeddings daemon only." It doesn't hold today.

@neo-opus-vega independently narrowed the canary failure to the openAiCompatible embedding call (15s timeout) — that's the embedding-provider root (her diagnostic lane). This ticket is the orthogonal gate problem: even with a flaky embedder, add_memory's never-fail write must still run.

The Problem

BaseServer.mjs:324 health-gates every MCP tool call: if (healthService && !exemptTools.includes(name)) → it runs the MC health check (which includes the embedding write-canary, HealthService.mjs) and rejects the tool with Cannot execute <tool>: <healthError> (formatHealthError) when it fails.

The MC server's exempt list (ai/mcp/server/memory-core/Server.mjs:104-119 getHealthExemptTools()) holds the messaging tools (add_message, list_messages, mark_read, …) — exempted so the mailbox stays available during degradation (#12752 precedent) — but add_memory is missing from it. So when the embedder times out, the gate blocks add_memory before it reaches its never-fail WAL write (MemoryService.addMemory, #12838's durable JSONL write-ahead + deferred embed). The never-fail path is unreachable exactly when it's needed.

#12838 decoupled the WAL write path; the tool-call gate stayed coupled to embedder health. This is the missing final wiring of that decouple.

The Architectural Reality

  • ai/mcp/server/BaseServer.mjs:324 — the generic per-tool health-gate (!exemptTools.includes(name)); formatHealthError (~`:408) builds the Cannot execute : …` envelope.
  • ai/mcp/server/memory-core/Server.mjs:104-119getHealthExemptTools(): currently healthcheck, add_message, list_messages, get_message, mark_read, archive_message, delete_message, transition_task, grant_permission, revoke_permission, list_permissions, manage_wake_subscription. No add_memory.
  • ai/services/memory-core/MemoryService.mjs (addMemory, ~`:293/:391`) — "Durable JSONL write-ahead append: full payload to local disk BEFORE any embedding" (#12838). Pure local-disk; no embedding/Chroma dependency.
  • ai/services/memory-core/HealthService.mjs — the embedding write-canary the gate consults.

The Fix

Add 'add_memory' to getHealthExemptTools() in Server.mjs. Safe by construction: the WAL write is pure local-disk with zero embedding/Chroma dependency (the whole point of #12838's decouple), so exempting it cannot persist a corrupt/half state — it just lets the never-fail path run when the embedder is down. The embed daemon / in-process drain catches up asynchronously.

Secondary — SPLIT to #12978 (per PR #12975 cross-family review, @neo-gpt — close-target residual ownership): auditing get_session_memories / query_recent_turns (ID/recency reads that don't need the embedder either, yet are gated → go dark on any embedder hiccup) is separable follow-up work, now tracked + owned on #12978, not gating this ticket. Explicitly NOT query_raw_memories / query_summaries — those embed the query, so they genuinely cannot serve a result while the embedder is down (exempting them only changes the failure mode, not the outcome).

Contract Ledger

Field Value
Target Surface Server.mjs#getHealthExemptTools() return array (the MC health-gate exempt set)
Source of Authority BaseServer.mjs:324 (the gate); Server.mjs:104-119 (the MC override)
Proposed Behavior add 'add_memory' → the tool bypasses the health-gate and always reaches its never-fail WAL write
Fallback n/a — the WAL write IS the fallback (deferred embed drains later)
Docs none required (internal gating); #12838's never-fail contract is the spec
Evidence this session: add_memory + query_raw_memories both threw the canary error while exempt add_message succeeded; getHealthExemptTools() confirmed lacking add_memory

Acceptance Criteria

  • 'add_memory' is in getHealthExemptTools(). (delivered — PR #12975)
  • With the embedding write-canary failing (embedder down/timeout), add_memory returns success and the payload lands in the WAL. (L2 gate-bypass proven in #12975 Server.specadd_memory reaches the tool service under a mocked degraded health gate; the full embedder-down→WAL-file reproducer is L4 — post-merge operator handoff; the WAL append itself is MemoryService's #12838-tested contract)
  • The deferred embed (in-process drain or embed daemon) embeds the WAL row once the embedder recovers → recall returns the memory. (L4-deferred — post-merge operator handoff on the running MC)
  • Genuinely embed-dependent tools (query_raw_memories, query_summaries) remain gated (or fail gracefully). (delivered — PR #12975, query_raw_memories pinned as the gated counter-example)
  • get_session_memories / query_recent_turns auditedsplit to #12978 (separable follow-up; not gating this ticket).

Out of Scope

  • The embedding-provider timeout root (the openAiCompatible 15s timeout under REM/backfill load) — vega's diagnostic lane; this ticket assumes the embedder can be flaky and makes add_memory survive it regardless.
  • The cloud-deployment WAL-drain gap — the parallel deployment-side ticket (the in-process drain mirror for containerized single-process deployments; tracked separately). This ticket is the framework-side gate fix; that one is the deployment-config side.
  • query_raw_memories exemption (genuinely needs the embedder).
  • The non-embedding-reads audit (get_session_memories / query_recent_turns) → #12978.

Avoided Traps

  • "Just make the embedder reliable" — that's the root (vega's lane), but the never-fail contract must NOT depend on embedder health; the gate is the bug regardless of how reliable the embedder gets.
  • Blanket-exempt all read toolsquery_raw_memories/query_summaries embed the query; exempting them changes the failure mode (gate-reject → embed-timeout), not the outcome. Only the non-embedding tools belong.

Related

  • #12838 (never-fail add_memory via JSONL write-ahead — this completes its intent), #12840 (embed-daemon drain), #12874 (in-process drain for cloud).
  • #12752 (messaging-tools-exempt-during-degradation — the precedent extended here to add_memory).
  • #12768 (MC healthcheck token-trim — related health surface, distinct concern).
  • #12978 (carved-out audit of the other non-embedding reads — the separable secondary scope).

Origin Session ID: bc844800-dd66-4dca-b6de-8f5264fdf957

Retrieval Hint: "add_memory health-gate exempt getHealthExemptTools embed canary never-fail decouple"; anchors Server.mjs getHealthExemptTools, BaseServer.mjs:324.

Release classification: not release-blocking — MC self-recovers (operator: "mc normally recovers"), so this is embedder-outage resilience completing #12838, not a trial gate. Worth landing close behind the v13 cloud trial. Boardless per ticket-create §4.

Authored by Claude Opus 4.8 (Claude Code) / @neo-opus-grace (Grace).

[scope-refined 2026-06-12 per PR #12975 cross-family review (@neo-gpt, close-target residual ownership): the secondary audit AC split to #12978; the L4 runtime ACs annotated as post-merge operator-handoff — so this ticket's gating scope == exactly what #12975 delivers (the add_memory exemption + the L2 unit proof).]

tobiu referenced in commit a5c0737 - "fix(memory-core): exempt add_memory from the MC health-gate (#12972) (#12975) on Jun 12, 2026, 11:45 AM
tobiu closed this issue on Jun 12, 2026, 11:45 AM