query_summaries empty: corrupt sessions vectors + swallowed errors

Context

The query_summaries Memory Core MCP tool returns {count:0, results:[]} for every semantic query — including plainly-matching ones ("DreamService SemanticGraphExtractor chunked Tri-Vector", "REM summarization chunking session", "Dream pipeline semantic graph extraction") — while its sibling query_raw_memories returns rich, relevant results for the identical queries. healthcheck reports the summaries collection neo-agent-sessions as count:1018 (930 team-shared), provider text-embedding-qwen3-embedding-8b @ 127.0.0.1:1234, status:healthy. So the collection is populated but semantic search over it is 100% non-functional, with no surfaced error.

A prior session flagged the same empty-summary-reads pattern and it was dismissed as a "transient glitch." This is not transient — it is a persistent, reproducible defect, and the reason it keeps getting mis-classified as transient is the second half of this ticket (a swallowed error).

The Problem

Verify-Before-Assert was applied before filing; the obvious hypothesis (embedding-model / dimension mismatch) was empirically falsified, and the real root cause was isolated to two layers.

Layer 1 — neo-agent-sessions vector segment is corrupt (the trigger)

The live Memory Core server log (mc-server-2026-06-03.log) shows the swallowed failure 14× today:

[WARN] [StorageRouter] Pass 1 semantic retrieval failed, falling back to empty result: Error executing plan: Internal error: Error finding id

A direct, read-only probe against the live ChromaDB (localhost:8000) characterizes the corruption precisely:

Interpretation: the metadata/record segment holds all 1018 summaries (count, get metadatas, list_summaries all work), but the HNSW vector segment is desynced — only ~68 of 1018 vectors are reachable via ANN search, and materializing stored embeddings throws Error finding id on unresolvable IDs. Dimension mismatch is ruled out (stored == query == 4096; the neo-agent-memory collection uses the same embedder and is fully healthy). Under the live server's concurrent write load, large-nResults summary queries traverse into the corrupt region and throw; query_raw_memories never throws because neo-agent-memory is intact. This is the asymmetry the operator observed.

Error finding id is a recurring ChromaDB-corruption signature across Memory Core's Chroma collections — previously seen on ask_knowledge_base (#12133, closed dropped; #11924 fallback-worked-around). It has been routed-around each time rather than root-caused.

Layer 2 — injectQueryReRanker swallows the error into a clean empty result (why it is invisible)

StorageRouter.injectQueryReRanker wraps both collections' query() and, on any Pass-1 throw, returns a clean empty result with only a logger.warn:

} catch (e) {
    logger.warn(`[StorageRouter] Pass 1 semantic retrieval failed, falling back to empty result: ${e.message}`);
    return {ids: [[]], distances: [[]], metadatas: [[]], documents: undefined};
}

The tool response ({count:0, results:[]}) is then indistinguishable from a genuine no-match. Consequences:

• A corrupt collection reads as "no results found" to every agent — masking the real failure for days and earning the "transient glitch" mis-diagnosis.
• healthcheck reports status:healthy + count:1018 because it only calls summaryCollection.count() (HealthService.mjs:869) and never runs a query canary — a collection that count()s but cannot query() passes the health check.

This swallow is the literal Bug-1 prescription from #9959 ("wrap Pass 1 ... with graceful fallback to empty result {ids: [[]]}"). That resilience patch — shipped without any upward observability — is now the mechanism hiding live corruption. This ticket is the friction→gold downstream of that decision, not a duplicate of it.

The Architectural Reality

• ai/services/memory-core/SummaryService.mjs:298 querySummaries() — calls collection.query({queryTexts:[query], include:['metadatas','documents']}). Code is correct; it faithfully queries a corrupt collection. (Its sibling MemoryService.queryMemories() at MemoryService.mjs:464 is structurally identical — same embedder, same re-ranker — and works only because neo-agent-memory is intact.)
• ai/services/memory-core/managers/StorageRouter.mjs:65-82 injectQueryReRanker() — the Pass-1 try/catch that converts a ChromaDB throw into a silent empty result. Injected on both memory and summary collections.
• ai/services/memory-core/managers/ChromaManager.mjs:221-234 getSummaryCollection() — both collections share one createDynamicTextEmbeddingFunction({providerResolver: () => aiConfig.embeddingProvider}), confirming embedding is identical across them.
• ai/services/memory-core/HealthService.mjs:864-873 — reports name/count for the session collection; no query-path probe.

The Fix

Two-layer, single coherent prescription:

1. Repair the data (immediate remediation). Rebuild the neo-agent-sessions HNSW/vector segment so all 1018 records are query-reachable — re-embed the summaries (the DreamService summary corpus is regenerable) or rebuild the collection index, or restore from a known-good backup. Verify post-repair that query_summaries returns non-empty for plainly-matching queries.
2. Close the observability hole (the durable, testable fix). Stop returning a clean empty result on a ChromaDB throw in injectQueryReRanker — surface the error (or a structured degraded marker) up to the tool response so "0 genuine matches" is distinguishable from "query path failed." Add a lightweight per-collection query canary (or a reachable-vector-vs-record-count divergence check) to HealthService so a populated collection whose query() throws or returns near-zero reachable vectors reports a non-healthy/degraded status instead of status:healthy.

Contract Ledger Matrix

Decision Record impact

none (no ADR asserted — not verified). The durable fix embodies the general "no silent fallback for a real failure" principle: a resilience fallback that erases the failure signal is worse than the crash it replaced, because it converts a loud, diagnosable error into a silent, multi-day-latent data-availability outage.

Acceptance Criteria

• neo-agent-sessions is repaired so all 1018 records are HNSW-reachable; query_summaries({query:'DreamService semantic graph extraction'}) returns non-empty. (ops/post-merge verification)
• injectQueryReRanker no longer returns a clean empty result on a ChromaDB throw — the error or a structured degraded marker reaches the query_summaries / query_raw_memories tool response. "0 matches" and "query failed" are distinguishable to the caller.
• healthcheck runs a per-collection query-canary (or reachable-vs-record-count divergence check) and reports a non-healthy/degraded status + structured field when a populated collection's query path throws or has near-zero reachable vectors.
• Regression test: a collection whose query() throws (Error finding id or any ChromaDB error) yields a surfaced error/degraded signal from query_summaries, not a silent {count:0}; and healthcheck reflects degraded for that collection.
• Document the corruption's origin (interrupted re-embed / migration / Chroma version event) so it does not silently recur; cross-link the prior Error finding id recurrences.

Out of Scope

• Changing the embedding provider/model or vector dimensions — V-B-A falsified the dimension-mismatch hypothesis (stored == query == 4096; neo-agent-memory healthy on the same embedder).
• #9959's Bug 2 (active-session summarization leak) — separate concern, tracked there.
• A general ChromaDB version upgrade (may be a cause to document, but the fix here is repair + observability, not an upgrade).

Avoided Traps

• Misdiagnosing as embedding-model/dimension mismatch — the intuitive first hypothesis. Falsified empirically: identical 4096-dim embedder, sibling collection healthy.
• "Just clear and re-add the summaries" and close it — fixes today's symptom but leaves the silent-failure mode that hid the corruption for days; it would recur invisibly. The observability fix is the load-bearing half.
• Treating it as a transient infra hiccup — the corruption is deterministic in an isolated probe (68/1018 reachable; get(embeddings) throws every time), not a flaky concurrent race.

Related

• #9959 — origin of the Pass-1 swallow (its Bug-1 prescription); OPEN. This ticket is its downstream observability consequence.
• #12133 — ask_knowledge_base type filter throws ChromaDB Error finding id (CLOSED, dropped) — sibling corruption signature.
• #11924 — prior Error finding id recurrence on ask_knowledge_base (fallback-worked-around).
• #12180 / #12331 — Chroma collection hygiene / orphan-collection cleanup (adjacent substrate).

Handoff Retrieval Hints

• Retrieval Hint: "query_summaries silently empty Error finding id neo-agent-sessions corrupt re-ranker swallow"
• Log signature: [StorageRouter] Pass 1 semantic retrieval failed, falling back to empty result: Error executing plan: Internal error: Error finding id
• Probe shape: direct ChromaDB query(queryEmbeddings, nResults:300) caps at 68/1018 on neo-agent-sessions; get(include:['embeddings']) throws.

Authored by Claude Opus 4.8 as @neo-opus-ada.