LearnNewsExamplesServices
Frontmatter
id9794
title[Memory Core] Fix UUID Graph Corruption and Stream Truncation Bug in REM Pipeline
stateClosed
labels
bugaicore
assigneestobiu
createdAtApr 8, 2026, 8:37 PM
updatedAtApr 8, 2026, 8:37 PM
githubUrlhttps://github.com/neomjs/neo/issues/9794
authortobiu
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtApr 8, 2026, 8:37 PM

[Memory Core] Fix UUID Graph Corruption and Stream Truncation Bug in REM Pipeline

Closedbugaicore
tobiu
tobiu commented on Apr 8, 2026, 8:37 PM

Architectural Adjustments for Autonomous REM Pipeline Stability

Context: The autonomous Memory Core processes legacy knowledge natively using headless background daemons (DreamService -> FileSystemIngestor -> GraphService). The reliability of the background memory ingestion pipeline hinges upon strict dataset determinism and payload parsing.

1. Deterministic Edge Topology (globalThis.crypto.randomUUID())

Issue: The SQLite Edge Graph Database utilized Neo.getId('edge') and Neo.getId('node') to assign relationship IDs. Because internal ID managers reset on every spin-up of the process execution block (e.g. testInferenceSpeed, runSandman), newly ingested edges inherited overlapping numerical IDs (e.g., e_1, e_2), triggering immediate corruption inside the memory database via overwrites.

Resolution: Transitioned all ID managers operating inside backend DB logic (Database.mjs, FileSystemIngestor.mjs, GraphService.mjs) to leverage V8's native UUID v4 Generator (globalThis.crypto.randomUUID()). Every edge entity now retains a fully pseudo-random ID ensuring safe ingestion regardless of Node environment.

2. Persistent Stream TCP Repair

Issue: Server-Sent Events (SSE) parsed by OpenAiCompatible.mjs frequently chunk data packets irregularly down the TCP network pipe. The parser evaluated streaming JSON output recursively, occasionally discarding mid-string chunked artifacts from the LLM, silently dropping REM extractions.

Resolution: Introduced an isolated line-length buffer to aggregate incomplete bytes across disjointed chunk borders consistently formatting the string parsing sequence for native JSON.parse().

tobiu added the bug label on Apr 8, 2026, 8:37 PM
tobiu added the ai label on Apr 8, 2026, 8:37 PM
tobiu added the core label on Apr 8, 2026, 8:37 PM
tobiu referenced in commit b52ef2f - "fix: UUID Graph Corruption and Stream Truncation Bugs in pipeline (#9794)" on Apr 8, 2026, 8:37 PM
tobiu assigned to @tobiu on Apr 8, 2026, 8:37 PM
tobiu
tobiu Apr 8, 2026, 8:37 PM

Input from Antigravity (Gemini 3.1 Pro):

✦ The structural identifiers and TCP chunking constraints have been resolved seamlessly across the primary engine architecture. Edges and Nodes use strictly determined universally unique identifiers via V8's native API. The LLM Stream utilizes buffered tracking to circumvent SSE chunking limitations across nodes.

Marking this Definition of Done as complete natively inside GitHub and closing!

tobiu closed this issue on Apr 8, 2026, 8:38 PM