LearnNewsExamplesServices
Frontmatter
id9914
titleEpic: Native Edge Graph Auditing and Deduplication Pipeline
stateClosed
labels
enhancementaiarchitecture
assigneestobiu
createdAtApr 12, 2026, 1:37 PM
updatedAtApr 12, 2026, 10:22 PM
githubUrlhttps://github.com/neomjs/neo/issues/9914
authortobiu
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[x] 9912 bug(ai): Prevent Vector Apoptosis from eradicating structural entities
blocking[ ] 9904 Epic: RLAIF Reward Function and Model Orchestration Pipeline
closedAtApr 12, 2026, 10:22 PM

Epic: Native Edge Graph Auditing and Deduplication Pipeline

Closed v13.0.0/archive-v13-0-0-chunk-3 enhancementaiarchitecture
tobiu
tobiu commented on Apr 12, 2026, 1:37 PM

Context

The SQLite memory matrix has rapidly scaled past 50,000 raw memory nodes. To stabilize RLAIF querying logic and reduce Context Bloat during RAG lookups, we must implement an automated database pruning daemon.

Scope

  • Cleanse duplicate string mappings.
  • Identify orphaned scalar CONTAINS edges.
  • Provide a vacuum / compact abstraction over the memory matrix to speed up memory-core operations.
tobiu added the epic label on Apr 12, 2026, 1:37 PM
tobiu added the ai label on Apr 12, 2026, 1:37 PM
tobiu added the architecture label on Apr 12, 2026, 1:37 PM
tobiu marked this issue as being blocked by #9912 on Apr 12, 2026, 1:40 PM
tobiu marked this issue as blocking #9904 on Apr 12, 2026, 1:40 PM
tobiu
tobiu Apr 12, 2026, 10:11 PM

Input from Antigravity (Gemini 3.1 Pro):

✦ ### 🔍 Defect Diagnosis: 19,500 Duplicate CONTAINS Edges Agent Antigravity actively investigated the Graph bloat issue regarding enormous amounts of orphaned scalar CONTAINS edges.

Root Cause Analysis: The duplicate edges are actively generated by FileSystemIngestor.mjs. When syncing the physical workspace structure to the Graph, the Ingestor creates CONTAINS edges from directories to files. To avoid duplicates, it uses: GraphService.db.edges.items.find(e => e.source === parentId && ...)

Because SQLite manages thousands of records natively, it does not load the full 36,000+ edge historical ledger into the db.edges.items RAM array. Therefore, .find() inevitably evaluates to false for existing structure edges, triggering recursive edge injection on every synchronization.

Proposed Resolution within this Ticket:

  1. Palliative Deduplication: Directly utilize SQLite's atomic abilities to DELETE duplicate mappings HAVING count(*) > 1.
  2. Structural Fix: Refactor FileSystemIngestor.mjs to bypass raw array .find() lookups. We will strictly route its edge insertion logic through GraphService.linkNodes(parentId, nodeId, 'CONTAINS', 1.0), which relies on synchronous SQL lookup checks (SELECT 1 FROM Edges WHERE...) to guarantee 100% deduplicated data constraints directly on the disk.
tobiu referenced in commit 0500d71 - "refactor(ai): Eliminate Edge topology duplicate bloat via native GraphService linking (#9914)" on Apr 12, 2026, 10:13 PM
tobiu cross-referenced by PR #9943 on Apr 12, 2026, 10:14 PM
tobiu assigned to @tobiu on Apr 12, 2026, 10:18 PM
tobiu removed the epic label on Apr 12, 2026, 10:19 PM
tobiu added the enhancement label on Apr 12, 2026, 10:19 PM
tobiu referenced in commit d3e1a9f - "refactor(ai): Eliminate Edge topology duplicate bloat via native GraphService linking (#9914) (#9943)" on Apr 12, 2026, 10:22 PM
tobiu closed this issue on Apr 12, 2026, 10:22 PM