DreamService Architecture Hardening & Memory Core Enhancements

Architectural Context

The Neo.mjs Memory Core and backend DreamService daemons require a deep refactoring to harden performance and inference reliability. Previous assumptions about Memory Core "context flooding" have been proven false: the episodic memory inherently stores dense cognitive data (prompt, thought, response) without raw boilerplate JSON streams, meaning structural token identifiers are properly maintained in the lowest levels.

Rationale

• Synchronous Bottlenecks: FileSystemIngestor.mjs and DreamService.mjs utilize heavy synchronous I/O, which block the Node.js event loop during intensive extraction operations.
• Tri-Vector Quality: DreamService currently synthesizes graph fragments from pre-summarized texts (via the summaryCollection), bleeding out precise framework identifiers (e.g., Neo.component.Base).
• Capability Gap Inference Flaws: p.includes(term) allows inaccurate sub-string matches (e.g., "List" matches "api_listener" or "whitelist").
• Race Conditions: sandman_handoff.md is updated synchronously without an atomic file swap, posing an application integrity risk if write operations clash.
• Cognitive Complexity Feedback: To accurately calculate output complexity during REM sleep, MemoryService needs mathematical inputs natively tracking agent expenditure (e.g., amountToolCalls and toolsUsed), rather than relying on LLM estimations during session summarization.

Proposed Changes

1. Async I/O: Refactor DreamService and FileSystemIngestor to utilize fs/promises.
2. Atomic Swaps: Leverage .tmp intermediate files and POSIX rename() for safe sandman_handoff.md generation.
3. Exact Matching: Migrate capability gap string checking to RegExp boundary assertions \b.
4. Tri-Vector Reality Check: Pipe the raw aggregatedContent from MemoryCollection directly into executeTriVectorExtraction instead of reading the generic session.document summary.
5. Memory Meta-Stats: Extend MemoryService.addMemory to natively accept amountToolCalls and an array of toolsUsed, enabling downstream SessionService productivity scoring.

Avoided Pitfalls

We explicitly avoid adding a mid-tier episodic summarization cascade before Tri-Vector ingestion, as doing so would actively destroy the localized structural Graph markers generated natively inside the agent's <thought> and response processes.