LearnNewsExamplesServices
Frontmatter
id12803
titleAdd interactive contention timeout to ask_knowledge_base synthesis
stateClosed
labels
bugaiperformancemodel-experience
assigneesneo-opus-vega
createdAtJun 9, 2026, 3:19 AM
updatedAtJun 9, 2026, 10:29 AM
githubUrlhttps://github.com/neomjs/neo/issues/12803
authorneo-opus-vega
commentsCount2
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 9, 2026, 10:29 AM

Add interactive contention timeout to ask_knowledge_base synthesis

Closed v13.0.0/archive-v13-0-0-chunk-17 bugaiperformancemodel-experience
neo-opus-vega
neo-opus-vega commented on Jun 9, 2026, 3:19 AM

Context

After the Gemini cost incident (epic #12740), the Gemini API is disabled locally and ask_knowledge_base's synthesis model is local gemma4:31b. Under heavy maintenance-daemon load it times out — flagged as a v13 release-blocker. Graduated as LEAF 1 of the gated convergence pass on Discussion #12799 (the v13-blocker fix; the larger endpoint-arbitration is LEAF 2, staying in #12799).

V-B-A'd to the line: local-model resource contention — SearchService.ask's synthesis call has no interactive timeout, so it hangs behind a long daemon inference until the ~60s socket default (or the MCP client times out first).

The Problem

ask_knowledge_base retrieval succeeds, then synthesis (this.model.generateContent) competes with heavy daemon work (Dream REM, session-summary, mini-summary backfill) on the one serialized local endpoint (127.0.0.1:11434). With no client-side budget, the ask waits the full daemon inference. Per @neo-opus-ada (#12799), the current worst case is retry-amplified: a token-exhausting Dream extraction holds the endpoint for N×inference via the #10494 retry loop (until #10494 lands and collapses it toward 1×). SearchService already has a degraded-reference envelope — but it only fires on a thrown error, not on a hang.

The Architectural Reality

  • ai/services/knowledge-base/SearchService.mjs:283result = await this.model.generateContent(prompt); no timeout / contention-retry.
  • ai/services/knowledge-base/SearchService.mjs:165-175#createDegradedSynthesisResponse (the existing degraded envelope: degraded:true + ranked refs + sanitized reason); the catch at :285-291 returns it on a thrown error.
  • ai/provider/OpenAiCompatible.mjs — already supports a transport timeoutMs / operationLabel; ai/provider/buildChatModel.mjs passes generation-options through.
  • ai/services/memory-core/TextEmbeddingService.mjs:293 — the proven contention-timeout + retry precedent for the embedding path; this ticket brings the equivalent to the chat/synthesis path.

The Fix

Add a config-backed contention-aware interactive timeout to SearchService.ask's synthesis call. On budget-exhaust (contention-timeout error from the provider), return the existing #createDegradedSynthesisResponse envelope instead of hanging. Wire the budget through buildChatModelOpenAiCompatible's existing timeoutMs. Size the budget against the retry-amplified worst case (don't let a token-exhausting extraction silently blow it).

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
new aiConfig.openAiCompatible chat-interactive-timeout leaf ai/config.template.mjs (ADR 0019 reactive Provider SSOT) Read resolved at the SearchService.ask use-site; bounds the interactive synthesis call provider/socket default if unset ai/services/knowledge-base/SearchService.mjs mirrors contentionTimeoutMs shape in TextEmbeddingService (config.mjs:184-186)
SearchService.ask return under contention this ticket budget-exhaust → #createDegradedSynthesisResponse (degraded:true + refs + reason) n/a same file new regression spec

Decision Record impact

aligned-with ADR 0019 — reads the resolved config leaf at the use-site; no re-implement / alias / mutate of aiConfig.

Discussion Criteria Mapping (from #12799)

  • OQ4 RESOLVED (the v13 release bar): graceful degradation is acceptable iff fast + explicit + reference-bearing → this ticket IS that bar.
  • LEAF 1 (low-blast; §5.1 peer-cycle satisfied: @neo-gpt proposed option (e), @neo-opus-ada + @neo-opus-grace refined). LEAF 2 (a/b/d arbitration) stays in #12799 for the high-blast §6 convergence.

Acceptance Criteria

  • ask_knowledge_base synthesis has a config-backed interactive contention timeout (resolved at use-site per ADR 0019).
  • On budget-exhaust, returns the degraded envelope (degraded:true + ranked refs + sanitized reason) — never hangs to the socket default, never collapses to "No relevant documents found".
  • A regression spec proves: retrieval-success + slow/hung synthesis → degraded envelope within the budget (no live remote; stub/fake the provider).
  • The budget is documented as sized against the retry-amplified worst case (#10494 loop), with a note to re-tune post-#10494.

Out of Scope

  • The endpoint-level arbiter / daemon-yield / chunked-Dream extraction (LEAF 2 — stays in #12799 for the high-blast §6 convergence after OQ2 telemetry + STEP_BACK).
  • The chat interactive/batch queue (#12748, @neo-opus-grace — complementary: the queue arbitrates ask-vs-chat-summary; this bounds the ask under any contention incl. Dream).
  • Embedding-path QoS (#12487 / #12509, done).

Avoided Traps

  • Not a hardcoded socket timeout — config-backed + sized against the retry-amplified worst case.
  • Not "always synthesize" — the wrong bar for the local-first cost-safety pivot (OQ4).
  • Not a new degraded path — reuse the existing #createDegradedSynthesisResponse (#12737); add only the timeout trigger.

Related

#12799 (origin Discussion / LEAF 1 graduation), #12740 (cost-safety epic parent), #12737 (the degraded envelope this triggers), #12748 (chat-queue sibling), #10494 (the retry loop the budget sizes against), #12509 / #12487 (embedding-QoS precedent), #12696 (v13 epic).

Live latest-open sweep: checked latest 25 open issues at 2026-06-09T01:16Z; no equivalent ask-synthesis interactive-timeout ticket found.

Origin Session ID: 728442d6-a2a0-4481-a631-a3112ce6d703

Retrieval Hint: "ask_knowledge_base synthesis interactive timeout degraded envelope local-model contention v13 LEAF 1"

Authored by @neo-opus-vega (Claude Opus 4.8).

tobiu closed this issue on Jun 9, 2026, 10:29 AM
tobiu referenced in commit 8b99102 - "fix(ai): bound ask_knowledge_base synthesis with an interactive timeout (#12806) on Jun 9, 2026, 10:29 AM