LearnNewsExamplesServices
Frontmatter
id14504
titleConcept-anchored retrieval: wrap the embedding path (consumer 1)
stateClosed
labels
enhancementaiarchitecture
assigneesneo-opus-vega
createdAtJul 2, 2026, 9:41 PM
updatedAt1:47 AM
githubUrlhttps://github.com/neomjs/neo/issues/14504
authorneo-opus-vega
commentsCount2
parentIssue14472
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAt1:47 AM

Concept-anchored retrieval: wrap the embedding path (consumer 1)

Closed Backlog/active-chunk-2 enhancementaiarchitecture
neo-opus-vega
neo-opus-vega commented on Jul 2, 2026, 9:41 PM

Context

Consumer-1 leaf of epic #14472 — the GraphRAG pattern over the self-extracted ontology, the epic's headline retrieval consumer. Filed in the operator-directed full-fledging pass; live latest-open sweep 2026-07-02T19:37:34Z: no equivalent ticket.

#14422 OQ5 routed the retrieval contract (wrap vs replace the embedding path) to the epic's measurement floor: the #14474 concept-neighborhood read probe supplies the reachability/provenance data that decides it. This leaf implements the wrap shape — the conservative fork — with the probe artifact as its go/no-go gate.

The Problem

23,487 durable memories and 1,411 session rollups have no concept-mediated retrieval path (graduation-record healthcheck). Retrieval today is flat embedding top-k: a query about delta-updates finds texts that sound like delta-updates, but never walks delta-updates → IMPLEMENTED_BY → src/vdom/Helper.mjs → attached memories/decisions — the structural neighborhood where the institution's actual knowledge of the concept lives. Consumer 4 (measurement) cannot compute re-derivation rates until retrieval events over concepts exist.

The Architectural Reality

  • Walk substrate: ai/services/memory-core/GraphService.mjs edge queries (the graduation's Matrix-1/B falsifier: metadata cannot walk; edges can).
  • Concept resolution: alias-aware AFTER the defrag leaf (#14502) lands its alias→canonical map; before that, resolution hits the fragmentation split (4 ids for Golden Path — #14474 scan).
  • Read-side annotations: the four-axis contract (#14422 OQ6, graduated) rendered per hop — authority (references shipped trustTier only) · fidelity · extractionProvenance (TAGGED_CONCEPT 1.0/0.8) · lifecycle. The #14474 probe establishes which axes are present in storage vs projection.
  • Consumer surface: the Memory-Core query tools (query_raw_memories / KB ask_knowledge_base) — a consumed-MCP-surface change, so the Contract Ledger below is binding and the OpenAPI/tool-description sites count per the NL/MCP wiring discipline.

The Fix

One PR, wrap-shape:

  1. Concept resolver: query → semantic top-k over CONCEPT-type nodes (+ alias-map lookup post-#14502) → resolved concept set with confidence.
  2. Bounded neighborhood walk: 1–2 hops per resolved concept (IMPLEMENTED_BY / PARENT_CONCEPT siblings / EXPLAINED_BY / attached memories-messages-decisions), hop budget + node-type filters explicit in config.
  3. Merged candidate set: embedding top-k ∪ walk-derived candidates, each walk candidate carrying provenance (via: concept-walk, the path, per-hop four-axis annotations) — the flat path stays untouched when the walk contributes nothing (wrap, never replace: embedding results are never displaced, only augmented).
  4. Opt-in flag on the tool surface (conceptWalk: true or equivalent) — default OFF for one release beat; flipping the default is a follow-up decision fed by consumer-4 measurements.
  5. Retrieval-event emission: each walk logs {query, resolvedConcepts, walkContributed, candidatesAdded} — the event stream consumer 4 (#14505-class measurement leaf) computes re-derivation rates from.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback / Edge Case Docs Evidence
query_raw_memories (MCP tool) #14422 OQ5 wrap ruling Optional conceptWalk param; response items gain optional via/conceptPath provenance Param absent → behavior byte-identical to today (spec-pinned) Tool description + OpenAPI site Compliance fixture + unit spec
KB ask_knowledge_base Native Edge Graph CONCEPT→FILE (guide/source) edges + the KB collection Opt-in conceptWalk: the KB FILE-resolver (buildKbFileResolveCandidate + QueryService.findDocBySource) walks CONCEPT→FILE edges → hydrates the KB doc whose metadata.source matches, reapplying the caller's own tenant and type filter (bc6f66f86); dedup by resolved source (ac528adaf) Param absent → byte-identical; graph unavailable / mid-walk read error → empty contribution (degrades to flat, never a query failure); FILE node with no matching/authorized KB doc → null (filteredOut); file:/file- id-dialects + flat duplicates deduped by resolved source Tool description + KB OpenAPI (pending cycle-2) Unit spec + tool-boundary MCP compliance fixture (pending cycle-2)
Walk module (ai/services/graph/) GraphService edge-query surface Read-only; hop/type budgets config-declared Graph unavailable → empty contribution, never a query failure Module docblock Hermetic spec on fixture graph

Decision Record impact

aligned-with ADR 0024 (read-only consumption). If the epic's OQ5 disposition later flips to REPLACE, that supersedes this leaf's wrap shape via a new decision — recorded on the epic, not silently.

Acceptance Criteria

  • Wrap-shape resolver + walk module + merged-candidate path implemented, opt-in, default OFF
  • Byte-identical default behavior spec-pinned (param absent ⇒ today's results)
  • Walk candidates carry via/path/four-axis provenance; absent axes render as absent (degrade-by-omission, the #14466-lineage honesty contract)
  • Retrieval events emitted per walk (the consumer-4 feed), schema documented
  • Hermetic specs: resolver, walk budgets, merge; MCP compliance fixtures updated (both tool sites)
  • Go/no-go gate honored: PR cites the #14474 artifact's reachability section; if reachability is too thin to add value, the leaf re-scopes to resolver-only and says so
  • PR Resolves this leaf, Refs #14472

Out of Scope

  • Replacing the embedding path (OQ5's other fork — epic-level decision).
  • Anchoring writes (OQ1 leaf #14506-class); default-ON flip (measurement-fed follow-up).
  • Cross-store federation (KB + MC walk merging beyond per-store enrichment).

Avoided Traps

  • Replace-by-stealth — wrap means the flat path survives byte-identical under the default; the spec pins it.
  • Unbounded walks — hop/type budgets are config, stated, and small; a graph walk must never dominate query latency (walk time logged per event).
  • Fabricated provenance — absent axis values render absent; the probe's absent-in-storage vs absent-in-projection distinction carries into the response shape.

Sequencing

Gated on the #14474 measurements artifact (go/no-go + axis-presence reality). Alias-aware resolution upgrades automatically when #14502 lands (soft dependency; resolver ships with normalize-only fallback).

Related: #14472 (parent epic), #14474 (gate), #14502 (alias map), Discussion #14422 (OQ5/OQ6)

Origin Session ID: 8cf234b7-e698-47ca-99e2-bf865196b6aa

Retrieval Hint: concept anchored retrieval wrap embedding neighborhood walk four-axis provenance opt-in

tobiu closed this issue on 1:47 AM
tobiu referenced in commit bb11e28 - "feat(ai): concept-anchored retrieval wrap — walk/merge core (#14504) (#15071) on 1:47 AM