Context
During v13 release work on 2026-06-08, ask_knowledge_base hard-failed while investigating the sync_all release-cache lane:
[GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1beta/models/gemini-3.5-flash:generateContent: [429 Too Many Requests]
The failure happened after the tool had already entered the RAG path, but the caller received only the provider exception. That matches the operator-reported MX friction that the ask tool was "WAY better before" and now regresses under current v13 load.
Live latest-open sweep: checked the latest 20 open issues at 2026-06-08T05:31Z (including author, title, labels, URL); no equivalent ticket. Closest current tickets are #12715 (lexical-rescue I/O), #12719 (candidate weighting), #12450 (Memory Core query_summaries Chroma error), and #12713 (miniSummary provider hang), but none cover SearchService.ask losing already-retrieved KB references when synthesis fails.
Duplicate sweeps:
- GitHub open issue search:
ask_knowledge_base quota fallback degraded synthesis -> no results.
- GitHub open PR search:
ask_knowledge_base quota fallback degraded synthesis -> no results.
- KB ticket semantic sweep via
query_documents(type='ticket') surfaced older generic KB/search issues, not an equivalent synthesis-provider fallback ticket.
- Exact corpus sweep over
resources/content/issues, resources/content/archive/issues, and resources/content/discussions surfaced related KB degradation history (#12450, #12133, #11924) and the ask-vs-query dominance note, but no issue that pins the synthesis quota/error fallback contract.
The Problem
ai/services/knowledge-base/SearchService.mjs retrieves ranked references before invoking the synthesis model:
QueryService.queryDocuments({query, type, limit, includeMetadata: true})
- Build
references
- Hydrate context
- Call
this.model.generateContent(prompt)
When step 4 fails with provider quota, transient provider outage, or missing/invalid generation model, the current implementation throws. The MCP caller loses the ranked KB evidence that was already available, forcing agents into a worse fallback path and hiding whether semantic search itself was healthy.
This is especially costly during v13 because agents are leaning on the KB for release-note evidence, code-location discovery, and friction triage under high provider load.
The Architectural Reality
- Owning service:
ai/services/knowledge-base/SearchService.mjs.
- Retrieval source of authority:
ai/services/knowledge-base/QueryService.mjs remains the ranked reference provider.
- Current tests live in
test/playwright/unit/ai/services/knowledge-base/SearchService.spec.mjs and already stub QueryService.queryDocuments plus SearchService.model.generateContent, making this regression cheap to cover without live Gemini.
- MCP consumer contract:
ask_knowledge_base is documented as the default Neo knowledge query because it returns a synthesized answer plus references. Under synthesis failure, it should degrade to references-plus-status, not erase the retrieval payload.
The Fix
Update SearchService.ask so synthesis-provider failures return a degraded response object instead of throwing after retrieval succeeds.
Expected shape:
- Keep returning
references built from QueryService.queryDocuments.
- Return an
answer that explicitly states synthesis failed and instructs the caller to use the references directly.
- Add a machine-readable degraded marker, e.g.
degraded: true, plus a bounded error / reason string that does not leak credentials.
- Preserve true retrieval failures as failures; this ticket only covers the post-retrieval synthesis path.
- Add focused unit coverage in
SearchService.spec.mjs for generateContent rejection after references exist.
Contract Ledger Matrix
| Target Surface |
Source of Authority |
Proposed Behavior |
Fallback |
Docs |
Evidence |
SearchService.ask() return payload |
Existing SearchService.ask references contract + MCP ask_knowledge_base tool description |
If retrieval succeeds but synthesis fails, return {answer, references, degraded: true, error} instead of throwing |
If retrieval itself fails, preserve the existing thrown failure path |
JSDoc / inline comments if return shape is widened |
Unit test stubs generateContent rejection and asserts references survive |
ask_knowledge_base MCP consumer behavior |
Tool description says ask returns synthesized answer plus refs and is the default Neo query tool |
Agents receive ranked refs even when Gemini quota/provider is exhausted |
Caller can still switch to query_documents manually if refs are insufficient |
No OpenAPI schema change unless the tool has a strict output schema |
Local unit test plus live manual recheck when provider recovers |
| Error disclosure |
AGENTS.md L2 / public artifact hygiene |
Error string is bounded and credential-safe; no API keys or raw env data |
Return generic synthesis_failed reason if message is unsafe |
PR body documents bounded behavior |
Unit test asserts no secret-shaped payload is required |
Decision Record impact
none. This is a service-level degradation-path bug fix aligned with the existing KB/MX direction; it does not amend an ADR.
Acceptance Criteria
Out of Scope
- Reworking
QueryService ranking or candidate weighting; that is covered by #12719.
- Lexical-rescue index performance; that is covered by
#12715.
- Root-causing Chroma
Error finding id / query_summaries corruption; that is covered by #12450.
- Switching generation providers or changing quota policy.
Avoided Traps
- Retrying Gemini inside
ask: rejected for this ticket. Retry/backoff can amplify quota exhaustion and still leaves callers blind if retries fail.
- Telling agents to use
query_documents instead: rejected. ask_knowledge_base is intentionally the primary tool; the tool should degrade well instead of pushing every caller into manual fallback.
- Catching all errors around retrieval + synthesis together: rejected. Retrieval failures are materially different from synthesis failures and must remain visible.
Related
Related: #12696
Origin Session ID: e8f07ef9-ef7e-4815-8ff4-7abe13720621
Retrieval Hint: "ask_knowledge_base synthesis provider quota fallback SearchService references degraded"
Context
During v13 release work on 2026-06-08,
ask_knowledge_basehard-failed while investigating thesync_allrelease-cache lane:The failure happened after the tool had already entered the RAG path, but the caller received only the provider exception. That matches the operator-reported MX friction that the ask tool was "WAY better before" and now regresses under current v13 load.
Live latest-open sweep: checked the latest 20 open issues at 2026-06-08T05:31Z (including author, title, labels, URL); no equivalent ticket. Closest current tickets are
#12715(lexical-rescue I/O),#12719(candidate weighting),#12450(Memory Core query_summaries Chroma error), and#12713(miniSummary provider hang), but none coverSearchService.asklosing already-retrieved KB references when synthesis fails.Duplicate sweeps:
ask_knowledge_base quota fallback degraded synthesis-> no results.ask_knowledge_base quota fallback degraded synthesis-> no results.query_documents(type='ticket')surfaced older generic KB/search issues, not an equivalent synthesis-provider fallback ticket.resources/content/issues,resources/content/archive/issues, andresources/content/discussionssurfaced related KB degradation history (#12450,#12133,#11924) and the ask-vs-query dominance note, but no issue that pins the synthesis quota/error fallback contract.The Problem
ai/services/knowledge-base/SearchService.mjsretrieves ranked references before invoking the synthesis model:QueryService.queryDocuments({query, type, limit, includeMetadata: true})referencesthis.model.generateContent(prompt)When step 4 fails with provider quota, transient provider outage, or missing/invalid generation model, the current implementation throws. The MCP caller loses the ranked KB evidence that was already available, forcing agents into a worse fallback path and hiding whether semantic search itself was healthy.
This is especially costly during v13 because agents are leaning on the KB for release-note evidence, code-location discovery, and friction triage under high provider load.
The Architectural Reality
ai/services/knowledge-base/SearchService.mjs.ai/services/knowledge-base/QueryService.mjsremains the ranked reference provider.test/playwright/unit/ai/services/knowledge-base/SearchService.spec.mjsand already stubQueryService.queryDocumentsplusSearchService.model.generateContent, making this regression cheap to cover without live Gemini.ask_knowledge_baseis documented as the default Neo knowledge query because it returns a synthesized answer plus references. Under synthesis failure, it should degrade to references-plus-status, not erase the retrieval payload.The Fix
Update
SearchService.askso synthesis-provider failures return a degraded response object instead of throwing after retrieval succeeds.Expected shape:
referencesbuilt fromQueryService.queryDocuments.answerthat explicitly states synthesis failed and instructs the caller to use the references directly.degraded: true, plus a boundederror/reasonstring that does not leak credentials.SearchService.spec.mjsforgenerateContentrejection after references exist.Contract Ledger Matrix
SearchService.ask()return payloadSearchService.askreferences contract + MCPask_knowledge_basetool description{answer, references, degraded: true, error}instead of throwinggenerateContentrejection and asserts references surviveask_knowledge_baseMCP consumer behaviorquery_documentsmanually if refs are insufficientsynthesis_failedreason if message is unsafeDecision Record impact
none. This is a service-level degradation-path bug fix aligned with the existing KB/MX direction; it does not amend an ADR.
Acceptance Criteria
SearchService.askreturns rankedreferenceswhenmodel.generateContent()rejects after retrieval succeeds.Out of Scope
QueryServiceranking or candidate weighting; that is covered by#12719.#12715.Error finding id/query_summariescorruption; that is covered by#12450.Avoided Traps
ask: rejected for this ticket. Retry/backoff can amplify quota exhaustion and still leaves callers blind if retries fail.query_documentsinstead: rejected.ask_knowledge_baseis intentionally the primary tool; the tool should degrade well instead of pushing every caller into manual fallback.Related
Related: #12696
Origin Session ID: e8f07ef9-ef7e-4815-8ff4-7abe13720621 Retrieval Hint: "ask_knowledge_base synthesis provider quota fallback SearchService references degraded"