Dedicated ask-synthesis model config + cost-safety guards

Context

ask_knowledge_base synthesis on the local gemma-4-31b was measured at 286.7s for a realistic 5-doc RAG prompt (11k input → 686 output tokens, this session) — unusable interactively, and it equals the 60s MCP SDK request-timeout ceiling (DEFAULT_REQUEST_TIMEOUT_MSEC). The swallow fix (#12831 → PR #12832) makes a timeout return graceful references, but synthesis still never completes. Operator direction: route ask synthesis to a fast remote model (Gemini Flash, ~5–10s) with its own dedicated config + cost-safety, while keeping a local provider as an option for privacy-sensitive deployments.

The Problem

• One global modelProvider serves all chat (ask / miniSummary / sessionSummary / graph). ask is interactive + needs a fast model; bulk summary/graph are high-volume + stay local (cost).
• The KB ask path's modelName is already gemini-3.5-flash, but the global provider was forced local (gemma, post-incident) so it ignores that and runs gemma → the 287s.
• Cost-safety is paramount. The prior cost incident was a scripted runaway (~1,200 calls/min × 40 min); interactive agents can't reach that volume. The remote path needs guards so it can't repeat.
• gemini-2.5-flash is 15× cheaper input / 22.5× cheaper output than 3.5-flash ($0.10/$0.40 vs $1.50/$9.00 per 1M; V-B-A'd via ai.google.dev) and sufficient for RAG synthesis → the right default.

The Architectural Reality

• ai/services/knowledge-base/SearchService.mjs construct() builds the model via buildChatModel({modelProvider: aiConfig.modelProvider, …, geminiModelName: aiConfig.modelName}) — the global provider, not an ask-specific one.
• ai/mcp/server/knowledge-base/config.template.mjs — the config home (already has askSynthesisTimeoutMs; modelName: 'gemini-3.5-flash').
• relates-to ADR-0019 (reactive Provider SSOT): extends it from one-global → a per-task block for the ask path.

The Fix

A dedicated askSynthesis block in knowledge-base/config.template.mjs; SearchService.ask resolves it, not the global modelProvider:

askSynthesis: {
    provider          : leaf('gemini',           'NEO_KB_ASK_PROVIDER'),   // gemini | openAiCompatible | ollama
    model             : leaf('gemini-2.5-flash', 'NEO_KB_ASK_MODEL'),
    // SECURITY: read ONLY from NEO_KB_ASK_API_KEY. NEVER inline here or in the generated config.mjs —
    // config.template.mjs is git-TRACKED (a key would ship to the repo); a secret on disk (even
    // gitignored config.mjs) risks accidental git-add-f / backups / leaks. Env is the secure channel.
    apiKey            : leaf(null,               'NEO_KB_ASK_API_KEY'),
    // Local-endpoint override for provider 'ollama'/'openAiCompatible' — set when the ask model runs on
    // its OWN endpoint (3-local-model setup: embed + summary + ask each on its own port). Unused for gemini.
    baseUrl           : leaf(null,               'NEO_KB_ASK_BASE_URL'),
    timeoutMs         : leaf(30000,              'NEO_KB_ASK_SYNTHESIS_TIMEOUT_MS', 'number'),
    maxCallsPerMinute : leaf(20,                 'NEO_KB_ASK_MAX_RPM', 'number')   // runaway breaker
}

3-layer cost-safety (the scripted-runaway class):

1. maxCallsPerMinute breaker — interactive use ≪ cap; a script trips it.
2. A dedicated ask API key (NEO_KB_ASK_API_KEY) → operator hard-caps the gcloud budget on just that key (the incident hammered a shared key; isolating it contains the blast radius).
3. gemini-2.5-flash itself — 22× cheaper output shrinks any blast.

Local-option preserved: provider→openAiCompatible|ollama + baseUrl→local → ask runs local (slow, no code leaves the box) for privacy-mandated deployments. The deploymentMode SSOT picks the preset.

Contract Ledger

Decision Record impact

relates-to ADR-0019 (extends the reactive Provider SSOT to a per-task ask block). Does not challenge it.

Acceptance Criteria

• askSynthesis block added to knowledge-base/config.template.mjs (+ regenerated config.mjs); SearchService.ask resolves it, not the global modelProvider.
• Default model: 'gemini-2.5-flash' (not 3.5); provider: 'gemini'.
• apiKey is env-only (NEO_KB_ASK_API_KEY) with the security comment; no inline value in template or config.mjs.
• baseUrl honored for ollama/openAiCompatible (local-endpoint override).
• maxCallsPerMinute breaker enforced on the ask remote path.
• Local-provider path works end-to-end (provider→openAiCompatible/ollama + baseUrl→local).
• Unit tests: provider resolution, the rate-cap breaker, env-only key (no inline), local-vs-remote routing.

Out of Scope

• The broader per-task-models + deployment-mode-presets architecture (miniSummary/sessionSummary/graph) → Ideation Discussion + epic.
• #12831/#12832's degraded-references swallow (shipped — stays the safety-net).
• The chat-model interactive/batch priority lane (#12748).

Avoided Traps

• Inline API key — rejected (security); env-only, the deliberate exception to "config.template defines values inline."
• Reusing the global modelProvider for ask — that's the bug (forces ask onto slow local gemma).
• Defaulting to 3.5-flash — 15–22× more expensive than 2.5-flash for no synthesis-quality gain here.
• Dropping local support — kept as the privacy escape hatch (no-remote-inference deployments).

Related

• #12740 — [Epic] Agent OS local-first AI provider defaults and cost-safety (parent)
• #12831 / #12832 — ask degraded-references swallow + the 287s measurement (the safety-net + the why)
• #12748 — chat-model interactive/batch priority lane (sibling)
• #12743 — Gemini incident spend reconciliation (cost-safety context)

Origin Session ID: 2feb15b9-1948-4553-9679-1419ed7eecf1 Retrieval Hint: "ask_knowledge_base dedicated askSynthesis config gemini-2.5-flash env-only NEO_KB_ASK_API_KEY maxCallsPerMinute runaway local-option" Authored by Claude Opus 4.8 as @neo-opus-grace.