LearnNewsExamplesServices
Frontmatter
id10151
titleDreamService: deterministic ingestion phase for Memory + Session graph nodes
stateClosed
labels
enhancementaiarchitecturecore
assigneestobiu
createdAtApr 21, 2026, 11:30 AM
updatedAtMay 15, 2026, 2:42 PM
githubUrlhttps://github.com/neomjs/neo/issues/10151
authortobiu
commentsCount0
parentIssue10143
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[x] 10153 Lazy back-fill: Memory + Session nodes for pre-migration Chroma rows, [x] 10152 SemanticGraphExtractor: emit typed edges to Memory + Session nodes
closedAtApr 21, 2026, 5:08 PM

DreamService: deterministic ingestion phase for Memory + Session graph nodes

Closed v13.0.0/archive-v13-0-0-chunk-5 enhancementaiarchitecturecore
tobiu
tobiu commented on Apr 21, 2026, 11:30 AM

DreamService: deterministic ingestion phase for Memory + Session graph nodes

Context

Parent sub-epic (#10143) phase 1: the deterministic Chroma-ID → graph-node ingestion step that runs before SemanticGraphExtractor on every REM cycle.

The Problem

Current REM pipeline treats memories/summaries as source material only. No anchor node exists in the graph. Downstream mailbox IN_REPLY_TO + identity AUTHORED_BY edges cannot proceed without these anchors.

The Architectural Reality

  • DreamService.processUndigestedSessions() runs phase 0 (ConceptIngestor), phase 1 (FileSystemIngestor), then per-session extraction (line 173-221).
  • No phase exists for ingesting the session/memory artifacts themselves.
  • Chroma IDs are stable; graph node IDs derive from them: memory:<chroma-id>, session:<chroma-id>.

The Fix

  • New method DreamService.ingestMemoryAndSessionNodes(session) — runs per session inside the REM loop, before executeTriVectorExtraction.
  • Emits Session node from session.meta (sessionId, title, createdAt, userId metadata).
  • Emits Memory nodes from StorageRouter.getMemoryCollection().get({where: {sessionId}}) — one node per memory, keyed by Chroma ID.
  • ORIGINATES_IN(Memory → Session) edges inline.
  • Graph schema: Memory and Session node types with properties {id, chromaId, sessionId?, title?, createdAt, userId?}. Indices on sessionId (Memory) and userId (both).

Acceptance Criteria

  • Graph schema migration adds Memory + Session node types (better-sqlite3 migration file)
  • ingestMemoryAndSessionNodes runs pre-extraction per session in REM pipeline
  • ORIGINATES_IN edges created for every Memory → Session
  • Re-run on same session is idempotent (no node duplication)
  • Graph integrity check: no orphan Memory nodes missing ORIGINATES_IN edge

Out of Scope

  • LLM-driven extraction (deterministic mapping only)
  • Back-fill of historical Chroma rows (separate sub-ticket)
  • Cross-tenant policy enforcement on nodes (under #10016)

Related

  • Parent: #10143 (Graph-first Memory artifacts sub-epic)
  • Unblocks: SemanticGraphExtractor enhancement, mailbox work (#10139), AgentIdentity ownership (#10016)
  • Files: ai/daemons/DreamService.mjs, graph schema migration

Origin Session ID: 71dc3cd8-d39d-48e1-ac62-e240ca67d1a5

tobiu added the enhancement label on Apr 21, 2026, 11:31 AM
tobiu added the ai label on Apr 21, 2026, 11:31 AM
tobiu added the architecture label on Apr 21, 2026, 11:31 AM
tobiu added the core label on Apr 21, 2026, 11:31 AM
tobiu added parent issue #10143 on Apr 21, 2026, 11:33 AM
tobiu marked this issue as blocking #10152 on Apr 21, 2026, 11:33 AM
tobiu marked this issue as blocking #10153 on Apr 21, 2026, 11:33 AM
tobiu cross-referenced by #10143 on Apr 21, 2026, 3:06 PM
tobiu cross-referenced by #10158 on Apr 21, 2026, 3:23 PM
tobiu marked this issue as blocking #10158 on Apr 21, 2026, 3:23 PM
tobiu assigned to @tobiu on Apr 21, 2026, 4:14 PM
tobiu cross-referenced by PR #10161 on Apr 21, 2026, 4:36 PM
tobiu referenced in commit c0cfb08 - "fix(ai): protect SESSION and MEMORY from getOrphanedNodes cleanup (#10151)" on Apr 21, 2026, 4:59 PM
tobiu referenced in commit 098a490 - "feat(ai): lift Memory + Session to first-class graph nodes (#10151) (#10161) on Apr 21, 2026, 5:08 PM
tobiu closed this issue on Apr 21, 2026, 5:08 PM
tobiu cross-referenced by #9999 on Apr 21, 2026, 7:03 PM
tobiu cross-referenced by PR #10171 on Apr 22, 2026, 12:47 AM
tobiu cross-referenced by #10172 on Apr 22, 2026, 1:19 AM