LearnNewsExamplesServices
Frontmatter
titlefeat: Harden DreamService memory processing and filesystem ingestion (#9884)
authortobiu
stateMerged
createdAtApr 11, 2026, 6:35 PM
updatedAtApr 11, 2026, 6:42 PM
closedAtApr 11, 2026, 6:41 PM
mergedAtApr 11, 2026, 6:41 PM
branchesdevagent/9884-dreamservice-hardening
urlhttps://github.com/neomjs/neo/pull/9885
Merged
tobiu
tobiu commented on Apr 11, 2026, 6:35 PM

Resolves #9884.

Fat Ticket Summary

This PR implements systematic hardening of the local Memory Core daemon layer:

  • Async Iterators: Dropped block-loop fs.readdirSync, readFileSync paths out of FileSystemIngestor.mjs, substituting complete fs/promises trees natively to unblock V8 event stack execution.
  • Atomic Renames: De-risked sandman_handoff.md persistence by forcing tmp-file streaming prior to isolated synchronous swapping, terminating dirty-read lock races across offline context loading.
  • Precise Boundary Recognition: Stripped lazy String.prototype.includes() overlap vectors out of Capability Inference mechanisms, instituting proper boundary \bREGEX\b checking.
  • Native Context Bindings: Enhanced the inner MemoryService models to actively cache amountToolCalls and toolsUsed parameter metrics directly into Chroma.

All local changes verify fully via memory graph Tri-Vector abstractions, scaling session complexity mapping transparently without inflating JSON prompt envelopes.

tobiu
tobiu commented on Apr 11, 2026, 6:40 PM

Input from Antigravity (Gemini 3.1 Pro):

✦ # PR Review Summary

Status: Approved

Thanks for putting this together! Great approach to solving the synchronous blocking issues in DreamService and FileSystemIngestor while simultaneously scaling session complexity mapping via the Chroma DB. The codebase hygiene here is stellar.


📊 Evaluation Metrics

  • [ARCH_ALIGNMENT]: 100 - Perfect alignment. Transitioning synchronous fs methods out of the execution path aligns correctly with Neo's async/multi-threading philosophy (even though this specific daemon runs natively on Node.js, V8 event loop unblocking is still paramount).
  • [CONTENT_COMPLETENESS]: 95 - The PR summary acts as a comprehensive "Fat Ticket" detailing architectural reasoning precisely.
  • [EXECUTION_QUALITY]: 95 - Implementing an atomic tmp-file rename for sandman_handoff.md avoids dirty reads elegantly. The transition to strict \b regex boundaries eliminates previous false overlap errors effectively.
  • [PRODUCTIVITY]: 100 - Achieves all stated goals. Telemetry is cleanly passed from parameters into the aggregation pipeline.
  • [IMPACT]: 85 - High impact for stability. Eliminating synchronous thread-blocking under high offline-memory capacity saves the daemon from latent failures.
  • [COMPLEXITY]: 75 - Heavy mix of standard library refactoring (fs/promises) alongside complex localized data layer mutation (Tri-Vector ingest flow mapping).
  • [EFFORT_PROFILE]: Architectural Pillar - Hardening daemon resilience directly impacts the swarm's future scaling capability.

🕸️ Context & Graph Linking

  • Target Epic / Issue ID: Resolves #9884
  • Related Graph Nodes: DreamService, FileSystemIngestor, MemoryCore, SessionService

🧠 Graph Ingestion Notes

  • [KB_GAP]: N/A - The implementation accurately reflects both standard Node async IO, the proprietary memory architecture schema, and OpenAPI specification dynamics natively.
  • [TOOLING_GAP]: It is notable that an autonomous agent recognized that modifying internal JS service parameters does not expose telemetry to the MCP envelope unless openapi.yaml is updated, demonstrating a deep integration competence.
  • [RETROSPECTIVE]: Returning the daemon read feed from pre-summarized data blocks back to raw Episodic Memory collections was a profound structural correction, as it preserves mathematical structural identifiers. Furthermore, utilizing atomic rename strategies (fs.promises.rename) over stream appends effectively bypasses complex filesystem polling race conditions.

📋 Required Actions

To proceed with merging, please address the following:

  • Integrate amountToolCalls and toolsUsed into openapi.yaml (Completed in recent commits).
  • Monitor CI pipelines.

Ready to squash merge upon successful CI results.