Context
V-B-A of @tobiu's concern (from the €50-incident follow-up): once we move summarization + mini-summaries + ask synthesis to the local gemma4 endpoint (#12742 + the #12740 epic), a heavy long-running maintenance request could block small interactive requests at the shared, serialized local endpoint.
The V-B-A confirms the worry and locates the gap precisely:
- The "small-request priority" logic that exists is the embedding interactive/batch queue (
ai/services/memory-core/TextEmbeddingService.mjs:127-202): runs local posts one-at-a-time "preferring interactive work" and explicitly "prevents add_memory from waiting behind a serialized batch."
- The chat path has no equivalent.
buildChatModel / SessionService summaries / MemoryService.buildMiniSummary / SearchService ask synthesis call the model with no interactive/batch lane.
- The heavy-maintenance lease (
MaintenanceBackpressureService) is task-level (coordinates the 7 heavy tasks; non-heavy work bypasses it at line 417) — it does not manage endpoint chat-request priority.
Today the block is masked because chat still hits remote Gemini (high concurrency). The local-models move removes that mask.
The Problem
A 30-60m heavy session-summary (or backfill) on the serialized local chat endpoint will queue ahead of a latency-sensitive interactive request (ask_knowledge_base synthesis; the mini-summary fired right after add_memory) → the interactive request waits until the heavy batch finishes / the MCP client times out. Exactly the failure the embedding queue was built to prevent — but for chat.
The Architectural Reality
- Pattern to mirror:
TextEmbeddingService.mjs:127-202 — #enqueueOpenAiCompatiblePost(inputData, options, priority) with priority: 'interactive'|'batch', FIFO-within-lane, interactive preferred.
- Chat call sites needing it:
SessionService.mjs buildChatModel (:38) + summarizeSessions/summarizePendingSessions; MemoryService.buildMiniSummary (:713); SearchService ask synthesis (:54).
- Endpoint reality: local OpenAI-compatible / Ollama servers serialize requests (TextEmbeddingService's own note).
The Fix
A shared chat-request interactive/batch lane on the local-endpoint path: interactive work (ask synthesis, post-add_memory mini-summary) preempts/queue-jumps batch work (heavy session-summary, backfill) on the serialized local endpoint. Mirror the embedding queue's semantics; classify heavy maintenance summaries as batch, user/agent-facing requests as interactive.
Decision Record impact
aligned-with ADR 0019 (reads provider config at use site; no mutation).
Acceptance Criteria
Out of Scope
- Embeddings (already covered —
TextEmbeddingService). Provider default → local (#12742). Backfill loop pacing (#12746 — that's the loop's rate, distinct from cross-request priority). KB-ask provider routing (#12741).
Related
- #12740 (epic). #12742 (the local-default move that makes this a prerequisite). #12746 (backfill loop-safety, sibling). €50 Gemini cost incident V-B-A follow-up, 2026-06-08.
Live latest-open sweep: per the 08:10Z cluster sweep + the 08:30Z merge-audit; no chat-priority-lane dup (distinct from the embedding queue + #12746's loop pacing).
Origin Session ID: 1cf2ad9f-28af-4242-9069-fd14c04e1b62
Retrieval Hint: query_raw_memories("chat request interactive batch priority lane local endpoint heavy summary block")
Authored by Claude Opus 4.8 (Claude Code), @neo-opus-grace.
Context
V-B-A of @tobiu's concern (from the €50-incident follow-up): once we move summarization + mini-summaries +
asksynthesis to the local gemma4 endpoint (#12742 + the #12740 epic), a heavy long-running maintenance request could block small interactive requests at the shared, serialized local endpoint.The V-B-A confirms the worry and locates the gap precisely:
ai/services/memory-core/TextEmbeddingService.mjs:127-202): runs local posts one-at-a-time "preferring interactive work" and explicitly "preventsadd_memoryfrom waiting behind a serialized batch."buildChatModel/SessionServicesummaries /MemoryService.buildMiniSummary/SearchServiceasksynthesis call the model with no interactive/batch lane.MaintenanceBackpressureService) is task-level (coordinates the 7 heavy tasks; non-heavy work bypasses it at line 417) — it does not manage endpoint chat-request priority.Today the block is masked because chat still hits remote Gemini (high concurrency). The local-models move removes that mask.
The Problem
A 30-60m heavy session-summary (or backfill) on the serialized local chat endpoint will queue ahead of a latency-sensitive interactive request (
ask_knowledge_basesynthesis; the mini-summary fired right afteradd_memory) → the interactive request waits until the heavy batch finishes / the MCP client times out. Exactly the failure the embedding queue was built to prevent — but for chat.The Architectural Reality
TextEmbeddingService.mjs:127-202—#enqueueOpenAiCompatiblePost(inputData, options, priority)withpriority: 'interactive'|'batch', FIFO-within-lane, interactive preferred.SessionService.mjsbuildChatModel(:38) +summarizeSessions/summarizePendingSessions;MemoryService.buildMiniSummary(:713);SearchServiceasksynthesis (:54).The Fix
A shared chat-request interactive/batch lane on the local-endpoint path: interactive work (
asksynthesis, post-add_memorymini-summary) preempts/queue-jumps batch work (heavy session-summary, backfill) on the serialized local endpoint. Mirror the embedding queue's semantics; classify heavy maintenance summaries asbatch, user/agent-facing requests asinteractive.Decision Record impact
aligned-with ADR 0019(reads provider config at use site; no mutation).Acceptance Criteria
batchsummarization in flight does NOT block aninteractivechat request (asksynthesis / post-add_memorymini-summary) on the local endpoint — interactive jumps the queue.batch; interactive ask/mini-summary classifiedinteractive.Out of Scope
TextEmbeddingService). Provider default → local (#12742). Backfill loop pacing (#12746 — that's the loop's rate, distinct from cross-request priority). KB-askprovider routing (#12741).Related
Live latest-open sweep: per the 08:10Z cluster sweep + the 08:30Z merge-audit; no chat-priority-lane dup (distinct from the embedding queue + #12746's loop pacing).
Origin Session ID: 1cf2ad9f-28af-4242-9069-fd14c04e1b62 Retrieval Hint:
query_raw_memories("chat request interactive batch priority lane local endpoint heavy summary block")Authored by Claude Opus 4.8 (Claude Code), @neo-opus-grace.