LearnNewsExamplesServices
Frontmatter
title>-
authortobiu
stateMerged
createdAtApr 15, 2026, 12:37 AM
updatedAtApr 15, 2026, 10:49 AM
closedAtApr 15, 2026, 10:49 AM
mergedAtApr 15, 2026, 10:49 AM
branchesdevagent/9999-optimize-local-summarization
urlhttps://github.com/neomjs/neo/pull/10019
Merged
tobiu
tobiu commented on Apr 15, 2026, 12:37 AM

Description

Resolves #10021

This PR removes the lossy map-reduce bottleneck and hard slicing that was destroying local inference performance for Gemma4 on the M5 processors. The map-reduce loop resulted in 30-60 min runtimes for 17 sessions.

Architectural Changes

  • Removed map-reduce chunking from SessionService summarization.
  • Removed slice(-3) from DreamService REM loop.
  • Updated SessionSummarization tests.
tobiu
tobiu commented on Apr 15, 2026, 10:46 AM

Input from Antigravity (Gemini 3.1 Pro):

✦ # PR Review Summary

Status: Approved

Thanks for putting this together! Great approach to stripping out the legacy map-reduce overhead. Sending the raw documents directly makes perfect sense since local models can now gracefully handle massive n_ctx capabilities via Flash Attention. This is ready to merge.


📊 Evaluation Metrics

  • [ARCH_ALIGNMENT]: 100 - Code successfully eliminates intermediate looping overhead, pushing the context parsing directly down to the LLM backend which fundamentally aligns with Neo's high-performance patterns.
  • [CONTENT_COMPLETENESS]: 90 - Testing explicitly asserts the map-reduce bypass by validating prompt count (toBe(1)) and length (> 20000).
  • [EXECUTION_QUALITY]: 95 - Removing slice boundaries enforces lossless abstraction. The <20s API duration test provides a strict contract against performance regressions.
  • [PRODUCTIVITY]: 100 - Directly addresses the 30-60 min latency bottleneck.
  • [IMPACT]: 80 - Solves a critical operational barrier for developers running the Agent OS locally.
  • [COMPLEXITY]: 30 - A targeted removal of complex code in favor of a simpler, native implementation.
  • [EFFORT_PROFILE]: Quick Win - High ROI for eliminating a massive local latency bottleneck by paring down logic.

🕸️ Context & Graph Linking

  • Target Epic / Issue ID: Resolves #10021
  • Related Graph Nodes: MemoryCore, SessionSummarization

🧠 Graph Ingestion Notes

  • [RETROSPECTIVE]: Removing artificial software abstraction (like Map-Reduce chunking) when the underlying foundational models natively support massive context windows yields immense performance multipliers. Always evaluate if the application layer algorithms are trying to solve capacity limits that the backend has already conquered.

📋 Required Actions

To proceed with merging, please address the following:

  • Code is clean. No further actions required.

LGTM. Ready for squash and merge into pending dev.