LearnNewsExamplesServices
Frontmatter
id13222
titleMemory query include omits distances → re-ranker loses semantic signal
stateClosed
labels
bugairegressionarchitecturemodel-experience
assigneesneo-opus-grace
createdAtJun 14, 2026, 2:05 PM
updatedAtJun 21, 2026, 2:17 AM
githubUrlhttps://github.com/neomjs/neo/issues/13222
authorneo-opus-grace
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 14, 2026, 2:35 PM

Memory query include omits distances → re-ranker loses semantic signal

Closed v13.1.0/archive-v13-1-0-chunk-2 bugairegressionarchitecturemodel-experience
neo-opus-grace
neo-opus-grace commented on Jun 14, 2026, 2:05 PM

Context

query_summaries and query_raw_memories (MCP) report distance: 0 / relevanceScore: 1 on every result — even when the underlying ranking is demonstrably correct (topically-relevant raw-memory results all came back at a perfect 1). Surfaced as one half of #12450 (the "swallowed errors" axis); my 2026-06-10 V-B-A on #12450 split it into two distinct bugs and flagged this score-reporting half as "a pure query-service reporting bug independent of the vector corruption — split it into its own leaf." This is that leaf.

Verified post-v13 (v13 shipped 2026-06-13); #12450 was dispositioned WAIVE→post-v13, so this is now in-scope.

The Problem

Both query paths pass an explicit chroma include array that omits 'distances':

  • ai/services/memory-core/MemoryService.mjs:1105include: ['metadatas']
  • ai/services/memory-core/SummaryService.mjs:374include: ['metadatas', 'documents']

ChromaDB returns only the fields named in include (ids always; distances only if 'distances' ∈ include). So the query ranks internally (correct order) but the response carries no distance valuessearchResult.distances is undefined.

The impact is larger than mis-reported scores. The Dual-Pass Re-Ranking Middleware (StorageRouter.injectQueryReRanker) consumes those distances as its Pass-1 semantic component:

  • StorageRouter.mjs:181 const distances = searchResult.distances?.[0] || [][]
  • :187 const vectorDist = Number(distances[index] ?? 0)0 for every candidate
  • :189 const semanticScore = 1 / (1 + vectorDist)1 (constant)
  • :209 compositeScore = semanticScore * topologyMultipliertopology-only (gravity + frontier weights); the semantic signal is silently eliminated

The re-ranker then re-packs distances: [topK.map(r => r.distance)] (:222) with the zeroed values, so the caller (MemoryService.mjs:1186 / SummaryService.mjs:452) computes distance:0 → relevanceScore:1 for every result.

Net: semantic search across the memory system silently degraded to topology-only ranking, with the perfect-1 score masking it as a healthy match.

The Architectural Reality

  • MemoryService.queryMemories (:1091) + SummaryService.querySummaries (:360) build queryArgs.include then call the re-ranked collection.query.
  • The re-ranker (StorageRouter.injectQueryReRanker, :126) wraps the chroma proxy's query, reads searchResult.distances?.[0], computes the composite score, and re-packs the chroma envelope.
  • Reference precedent: the watermark probe (StorageRouter.mjs:90) calls query() with no explicit include (chroma's default returns distances), confirming the explicit include is what suppresses them.

The Fix

Add 'distances' to both include arrays:

  • MemoryService.mjs:1105: ['metadatas']['metadatas', 'distances']
  • SummaryService.mjs:374: ['metadatas', 'documents']['metadatas', 'documents', 'distances']

Restores real vector distances end-to-end: Pass-1 distances → real semanticScore → real composite ranking → real re-packed distances → correct relevanceScore.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
query_raw_memories[].relevanceScore / .distance MemoryService.mjs:1186-1187 reflect real chroma vector distance (varies, not constant 1) n/a (always present) MCP tool schema regression test: distinct embeddings → distance>0, relevanceScore<1
query_summaries[].relevanceScore / .distance SummaryService.mjs:452-453 same n/a MCP tool schema shared mechanism; summary-path include parity
Dual-Pass re-ranker semantic component StorageRouter.mjs:187-189 non-constant semanticScore driving composite rank topology-only (the current bug) re-ranker JSDoc distances flow restores semantic signal

Decision Record impact

none — no ADR governs the chroma include params; this is a service-layer correctness fix.

Acceptance Criteria

  • MemoryService.queryMemories and SummaryService.querySummaries include 'distances' in the chroma include.
  • A regression test asserts real (non-constant) relevance: distinct embeddings → at least one result with distance > 0 / relevanceScore < 1 (the current bug yields a constant 1). The pre-existing relevanceScore > 0 assertion is insufficient — 1 > 0 passes the bug.
  • Re-ranker semantic signal restored (non-zero distances feed semanticScore), not topology-only.

Out of Scope

  • Bug (a) — summary-vector corruption / query_summaries recency-fallback (#12450's other half): if the neo-agent-sessions summary vectors are independently corrupt/missing, this distances-fix restores score reporting but the summary ranking may still be wrong. That couples to #12828 (scheduler/backfill) + #12830 (corrupt-vector root-cause). Post-merge validation will determine whether this leaf also resolves the summary symptom or only the reporting.
  • Any change to the re-ranker's topology weighting or the embedding provider.

Related

  • Parent: #12450 (this is the bug-(b) leaf, split out per my 2026-06-10 analysis).
  • Sibling (bug-a coupling): #12828, #12830.

Release classification: post-release (memory-core query correctness; v13 already shipped 2026-06-13 — not v13.x-blocking).

Origin Session ID: 0f5d9f1d-0683-452d-aac1-f467297186ac

Retrieval Hint: "chroma query include omits distances re-ranker semantic score relevanceScore 1"

Authored by Claude Opus 4.8 (Claude Code), @neo-opus-grace (Grace).