LearnNewsExamplesServices
Frontmatter
titlefeat: Two-Pillar Hybrid RAG Architecture (#9922)
authortobiu
stateMerged
createdAtApr 12, 2026, 4:23 PM
updatedAtApr 12, 2026, 5:36 PM
closedAtApr 12, 2026, 5:36 PM
mergedAtApr 12, 2026, 5:36 PM
branchesdevagent/9922-two-pillar-rag
urlhttps://github.com/neomjs/neo/pull/9926
Merged
tobiu
tobiu commented on Apr 12, 2026, 4:23 PM

Resolves #9922 Resolves #9923 Resolves #9924 Resolves #9925

The Two-Pillar Hybrid RAG Architecture

This PR transitions the Neo.mjs memory core from a bottlenecked O(N) brute-force search to a scalable, high-performance Two-Pillar Hybrid RAG architecture.

Architectural Accomplishments

  1. Engine Decoupling & Config: Updated the central configurations (aiConfig.engine = 'hybrid') and re-built the SystemLifecycleService to enforce an external-daemon-first boot architecture, prioritizing ChromaLifecycleService.
  2. Dual-Diagnostics HealthService: Refactored HealthService.mjs to separately validate and assert Chroma network accessibility alongside SQLite Native Edge Graph file integrity, ensuring granular fail-states.
  3. Database Topology Extrications: Added native graph export/import via JSONL in DatabaseService.mjs and isolated db truncation tools so Memory Core agents can prune their episodic vs structural knowledge independently.
  4. Re-Ranking Middleware (The Two-Pillar Core): Extricated the heavy and blocking SQLiteVectorManager.mjs completely. Injected a Dual-Pass Re-Ranking Middleware into StorageRouter.mjs.
    • Pass 1: It queries ChromeDB using vectorized ANN search to extract top-K candidates (nResults * 3).
    • Pass 2: It leverages synchronous structural pulls against the SQLite Native Edge Graph, boosting semantic rank based on Active Context Frontier connectivity and Graph Node Gravity (inward/outward degrees) before re-slicing Top K.

Avoided Pitfalls

  • Constructing V8 HNSW indexes via JSON SQLite data parsing forces colossal garbage collection. Bypassing JS parsing and utilizing true ChromaDB + local SQLite Graph structures completely eliminates engine freezes under heavy topological searches.
tobiu
tobiu commented on Apr 12, 2026, 5:35 PM

Input from Gemini 3.1 Pro (Antigravity):

✦ # PR Review Summary

Status: Approved

Autonomous PR Review completed. The code accurately aligns with the "Fat Ticket" objective to establish the Two-Pillar Hybrid RAG Architecture and cleanly intercepts the memory retrieval pathways without destructive Garbage Collection overhead.


📊 Evaluation Metrics

  • [ARCH_ALIGNMENT]: 100 - Perfectly aligns with the Two-Pillar Hybrid RAG architectural directive, cleanly isolating vector logic while utilizing local edge graphs synchronously.
  • [CONTENT_COMPLETENESS]: 100 - "Fat Ticket" protocol is honored. The PR body is comprehensive. New middleware methods contain complete JSDoc defining the dual-pass logic.
  • [EXECUTION_QUALITY]: 100 - StorageRouter securely intercepts the CollectionProxy without polluting upstream services. Safe fallbacks for empty vectors implemented.
  • [PRODUCTIVITY]: 100 - All components defined in the #9922 Epic and its sub-tasks directly resolved.
  • [IMPACT]: 90 - Critical framework evolution offloading heavy computations from memory-core SQLite graphs to ChromaDB.
  • [COMPLEXITY]: 75 - Handled lifecycle reordering, diagnostic splits, and query injections smoothly.
  • [EFFORT_PROFILE]: Architectural Pillar - Fundamental architectural shift in Memory Core scaling limits.

🕸️ Context & Graph Linking

  • Target Epic / Issue ID: Resolves #9922, #9923, #9924, #9925
  • Related Graph Nodes: memory-core, hybrid-rag, StorageRouter

🧠 Graph Ingestion Notes

  • [RETROSPECTIVE]: Replacing SQLiteVectorManager with the Dual-Pass StorageRouter middleware bypasses parsing huge JSON payloads in V8, virtually eliminating GC freezing. The real-time mapping of structural node gravity alongside Chroma ANN search creates highly context-aware memory retrievals seamlessly.

📋 Required Actions

To proceed with merging, please address the following:

  • None. Code is validated and JSDoc meets the contextual completeness gate.

Ready to merge utilizing the Squash Merge strategy.