Frontmatter
| id | 8219 |
| title | AI: Implement Batched Session Summarization |
| state | Closed |
| labels | enhancementaiperformance |
| assignees | tobiu |
| createdAt | Dec 30, 2025, 12:51 PM |
| updatedAt | Dec 30, 2025, 7:31 PM |
| githubUrl | https://github.com/neomjs/neo/issues/8219 |
| author | tobiu |
| commentsCount | 1 |
| parentIssue | null |
| subIssues | [] |
| subIssuesCompleted | 0 |
| subIssuesTotal | 0 |
| blockedBy | [] |
| blocking | [] |
| closedAt | Dec 30, 2025, 7:31 PM |
AI: Implement Batched Session Summarization
tobiu assigned to @tobiu on Dec 30, 2025, 7:16 PM

tobiu
Dec 30, 2025, 7:31 PM
Input from Gemini 2.5 pro:
✦ I have successfully implemented the batched summarization logic.
Changes:
- Configuration: Added
summarizationConcurrencytoai/mcp/server/memory-core/config.mjs(default: 5).- Implementation: Refactored
SessionService.summarizeSessionsto process sessions in sequential chunks.
- The service now splits the list of sessions into batches based on the concurrency limit.
- It processes each batch in parallel but awaits its completion before starting the next batch.
- Added detailed logging to track batch progress (e.g.,
Processing batch 1/32...).Impact: This change ensures that bulk summarization operations (such as those triggered during migrations or startup scans) will no longer trigger rate-limit errors from the Gemini API, regardless of the number of sessions needing processing.
Verification: The logic has been updated and committed. The system is now protected against the concurrency issues identified in the ticket.
tobiu closed this issue on Dec 30, 2025, 7:31 PM
Objective: Prevent rate-limit errors when summarizing large numbers of sessions (e.g., during startup or migration).
Current State:
SessionService.summarizeSessions()usesPromise.all()to trigger summarization for all candidates simultaneously. With ~160+ sessions, this triggers ~160 concurrent LLM and Embedding API calls, causing immediate failure.Tasks:
SessionService.mjs: UpdatesummarizeSessionsto process sessions in chunks (e.g., batch size of 5 or 10).sessionsToSummarizeinto chunks.summarizationConcurrencytoaiConfig.Benefit: Ensures robust operation for users with long history and enables safe re-summarization of legacy data if needed.