LearnNewsExamplesServices
Frontmatter
id9921
titlefix(ai): Resolve local inference 4096 n_ctx Exhaustion During Session Summarization
stateClosed
labels
bugai
assigneestobiu
createdAtApr 12, 2026, 3:25 PM
updatedAtApr 14, 2026, 12:32 AM
githubUrlhttps://github.com/neomjs/neo/issues/9921
authortobiu
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[ ] 9954 Epic: The Self-Healing Protocol
closedAtApr 13, 2026, 1:54 PM

fix(ai): Resolve local inference 4096 n_ctx Exhaustion During Session Summarization

tobiu
tobiu commented on Apr 12, 2026, 3:25 PM

Description

The mcp_neo-mjs-memory-core_summarize_sessions tool routinely crashes internally with the local inference daemon error: The number of tokens to keep from the initial prompt is greater than the context length (n_keep: 5382 >= n_ctx: 4096).

This indicates that as Neo.ai.Agent reasoning loops expand during heavily contextual RLAIF sessions (due to increased swarm intelligence and tool use), the final session summarization pipeline strictly drops the payload and fails to index the session.

Architectural Rationale

  • The underlying backend (e.g., MLX / openAiCompatible) currently defaults to a context limit of 4096 tokens when hosting gemma4:31b or similar local models.
  • We must either explicitly bump this n_ctx limit at the deployment daemon layer (e.g., updating Python boot args to 8192), or we must instruct the memory-core MCP Server to truncate/chunk the combined prompt + thought + response strings before throwing the batch at the extraction endpoint.
  • A2A Context: Failure to resolve this natively prevents the most complex agent problem-solving sessions from being squashed into Vector Summaries. Consequently, the Swarm loses immediate visibility to precisely the problems it struggled with the most.

Origin Session ID: 95bf4a2b-d84e-4f70-945b-f558ba924d3a

tobiu added the bug label on Apr 12, 2026, 3:25 PM
tobiu added the ai label on Apr 12, 2026, 3:25 PM
tobiu marked this issue as blocking #9954 on Apr 13, 2026, 1:13 PM
tobiu referenced in commit 002d2f0 - "fix(MemoryCore): Resolve local MLX n_ctx exhaustion by truncating tail-end session documents (#9921)" on Apr 13, 2026, 1:43 PM
tobiu cross-referenced by PR #9964 on Apr 13, 2026, 1:43 PM
tobiu cross-referenced by #9965 on Apr 13, 2026, 1:53 PM
tobiu referenced in commit c4a9eda - "fix(MemoryCore): Resolve local MLX n_ctx exhaustion by truncating tail-end session documents (#9921) (#9964)" on Apr 13, 2026, 1:54 PM
tobiu closed this issue on Apr 13, 2026, 1:54 PM
tobiu cross-referenced by PR #9966 on Apr 13, 2026, 1:58 PM
tobiu assigned to @tobiu on Apr 14, 2026, 12:32 AM