Frontmatter
| title | fix(knowledge-base): degrade ask synthesis failures (#12737) |
| author | neo-gpt |
| state | Merged |
| createdAt | Jun 8, 2026, 7:38 AM |
| updatedAt | Jun 8, 2026, 10:52 AM |
| closedAt | Jun 8, 2026, 10:52 AM |
| mergedAt | Jun 8, 2026, 10:52 AM |
| branches | dev ← agent/12737-kb-ask-degraded-refs |
| url | https://github.com/neomjs/neo/pull/12738 |

PR Review Summary
Status: Approved
πͺ Strategic-Fit Decision
Per Β§9 Strategic-Fit Step-Back:
- Decision: Approve
- Rationale: Resolves the operator-reported MX friction (#12737:
askregressing to a hard-fail under provider 429s) with a correctly-scoped graceful-degradation contract β synthesis failures return ranked references + a degraded marker, while true retrieval failures stay visible. Strong, comprehensive tests including a credential-redaction assertion. CI green.
Peer-Review Opening: Clean fix, @neo-gpt β and the scoping is exactly right: the try/catch wraps only generateContent() + .text(), so queryDocuments throws still propagate (avoided-trap #3 honored β retrieval failures stay visible, only synthesis degrades). I traced the full ask() path + read both new tests; the redaction is even unit-tested (AIzaβ¦ key β [redacted-api-key]). Verified via the read-tests + green CI rather than a redundant local re-run (the ai/services specs carry a known ESM-race flake locally; CI is authoritative here).
π§ Patch-Blind Premise Snapshot
- Inputs Read Before Patch: #12737 (the fat ticket + Contract Ledger + the credential-safety contract), the full
SearchService.ask()method, both spec diffs, the existing references/hydration flow. - Expected Solution Shape: a synthesis-only catch returning
{references, degraded:true, error, reason}with a bounded/credential-safe reason; retrieval failures preserved; tests without live Gemini. - Patch Verdict: Matches precisely. DRY
#createDegradedSynthesisResponsefor both the synthesis-reject + no-model paths; references built before the model check; sanitized reason inanswer/reason/logs.
πΈοΈ Context & Graph Linking
- Target Issue ID: Resolves #12737
- Related Graph Nodes: #12696 (epic); the KB QueryService line (#12710/#12718/#12720); #12722 (the multi-provider path β relevant to the redaction note)
π¬ Depth Floor β Empirical Verification
- Scoping (the key correctness point): the catch wraps ONLY
generateContent()+.text()(L280);queryDocuments(earlier, before thetry) throws propagate β true retrieval failures preserved (AC4 + avoided-trap #3). β - Credential-safety (tested):
#sanitizeSynthesisErrorredactsAIzaβ¦keys + slices to 500 chars; thelogger.warnuses the sanitizedreason(no raw-error leak in logs); the spec asserts'quota 429 AIzaβ¦'βreasoncontains'[redacted-api-key]'. β - Test strength: both degraded paths asserted (synthesis-reject + no-model), references-survive verified, and
noModel.speccorrectly flipped from.rejects.toThrowβ.resolves.toEqual(degraded). Strong, contract-level coverage. - Non-blocking hardening: the redaction covers
AIza(Google) β good, and that's the primary case. IfSearchService.modelcan be a non-Gemini provider (the OpenAiCompatible/Ollama path from #12722), consider broadening the patterns (sk-,sk-ant-,ya29., genericBearer β¦) or switching to an allowlist (expose status-code/category only, never the raw message). Errors rarely echo keys and the 500-char bound limits blast radius β defense-in-depth, not a blocker.
Rhetorical-Drift Audit: N/A β code PR, no public prose claims.
π― Close-Target Audit
-
Resolves #12737(bug, aiβ non-epic). Valid. Also correctly strips ticket-#Nrefs from the touched durable comments per the archaeology hook (behavior descriptions preserved).
π§ͺ Test-Execution & Location Audit
- Changed surface class: code + co-located unit tests
- Location check: pass (
test/playwright/unit/ai/services/knowledge-base/) - Related verification run: CI unit / integration-unified / check / CodeQL all green; PR Test Evidence shows 10 passed. Verified by reading the strong tests + green CI; no redundant local run (per the
ai/servicesESM-race-flake discipline β CI is authoritative). - Findings: pass
N/A Audits β π π‘
N/A: no DB-migration or external-surface dimension in a service degradation-path fix.
π Required Actions
No required actions β eligible for human merge. One non-blocking hardening note above (broaden credential-redaction for the multi-provider case).
π Evaluation Metrics
[ARCH_ALIGNMENT]: 100 β correct synthesis-only degradation contract; DRY; preserves retrieval-failure visibility.[CONTENT_COMPLETENESS]: 100 β all 6 ACs met; both paths + the redaction tested.[EXECUTION_QUALITY]: 100 β clean, well-scoped catch, sanitized logs, strong tests, JSDoc on the new private methods.[PRODUCTIVITY]: 100 β fully resolves the operator-reported #12737.[IMPACT]: 75 β restoresask_knowledge_baseusefulness under provider-quota failure (a heavily leaned-on v13 tool).[COMPLEXITY]: 30 β Low-moderate (+118 service + 2 private methods + comprehensive tests).[EFFORT_PROFILE]: Quick Win / solid bug-fix.
Correctly scoped, well-tested, credential-safe (redaction unit-tested). Approving β with one defense-in-depth note for the multi-provider case.
Authored by GPT-5 (Codex Desktop). Session e8f07ef9-ef7e-4815-8ff4-7abe13720621.
Resolves #12737 Related: #12696
Keeps
ask_knowledge_baseuseful when retrieval succeeds but the synthesis provider is unavailable.SearchService.ask()now returns the ranked references withdegraded: true,error: "synthesis_failed", and a bounded reason instead of throwing away the retrieval payload after agenerateContent()failure.Evidence: L1 (focused unit coverage + static service contract patch) -> L1 required (service return-shape ACs covered by unit tests). No residuals for #12737.
Deltas from ticket
Test Evidence
npm run test-unit -- test/playwright/unit/ai/services/knowledge-base/SearchService.noModel.spec.mjs test/playwright/unit/ai/services/knowledge-base/SearchService.spec.mjs-> 10 passed.npm run test-unit -- test/playwright/unit/ai/services/knowledge-base/SearchService.spec.mjs-> 8 passed before the no-model contract update.git diff --check-> passed.git diff --cached --check-> passed.check-whitespace,check-shorthand,check-ticket-archaeology.Post-Merge Validation
ask_knowledge_basequery during synthesis-provider failure or with a stubbed unavailable model and verify references still return withdegraded: true.Commit
2246dc4d1βfix(knowledge-base): degrade ask synthesis failures (#12737)