ask_knowledge_base needs a chat instance that cannot collide, and a model chosen by measured speed AND quality

Lane goal

ask_knowledge_base is usable from every agent seat, on a serving endpoint that no other workload can contend with.

Demonstrable bar: a seat calls the tool with no special parameters, gets a cited answer well inside the client deadline, and a concurrent session summary or graph pass cannot change that.

Problem scope

AGENTS.md §edge_case_triggers mandates this tool as the first step for Neo concepts. It does not work at its shipped default, so following the rule as written produces abandonment — measured across the fleet as five peers, zero calls since the provider switch.

Measured on the local plane, deployedRevision 271bb132d8, with no maintenance task running (verified before measuring — this is the best case, not the bad one):

Two causes, and neither is "the model is too big".

1. No request carries a cost bound. SearchService.ask() → QueryService.queryDocuments({limit}) reads each hit's whole file and joins them:

const contextDocs = (await Promise.all(contextPromises)).join('\n\n');

No truncation, no character budget, no token cap on that path. Cost is decided by whatever ranks top-limit — context-engineering-done-right.md is 18,800 tokens by itself, so limit is a proxy for cost, not a bound on it, and changing the default moves the cliff without removing it. Meanwhile askSynthesis.timeoutMs defaults to 300,000 ms while the measured seat abandons below 43,000 ms: the path keeps working roughly seven times longer than any caller waits. That is #16706's FIX-1 — never dispatch a request we cannot cancel and will not wait for — violated by a bound answerable to no caller, not by a missing one.

2. One serving endpoint answers every chat consumer. The dedicated config seam already exists and is shipped (#12836): askSynthesis owns provider, model, apiKey and baseUrl at ai/mcp/server/knowledge-base/configBase.mjs:206, SearchService builds its model from that block and not the global one, and Compose carries NEO_KB_ASK_* into KB. askSynthesis.baseUrl defaults to null, which resolves to the provider's default host — so ask, summarisation and graph work land on the same server. The seam is present and unbound.

Corrected after @neo-gpt-emmy's Stage-2 review (#16998 Stage-2, verified independently at dev 271bb132d85b8219d9565c201ea198f39d99029a): this scope previously blamed ai/provider/buildChatModel.mjs for cross-service contention. Its sharedLocalChatRequestQueue is process-wide, and KB and MC are separate processes, so it serializes within a service and is not the mechanism. Collision happens at the shared endpoint, which makes isolation a deployment property, not a code one.

Intended solution shape

Hard isolation, operator-set and non-negotiable: a chat-model instance used ONLY by the ask tool, which never collides with any other chat request. Session summaries and graph/Dream processing keep the full-power model; the lanes are separated, never traded off.

The load-bearing reason is scalability, not tidiness (operator, relayed via @neo-gpt-emmy). A full-power Dream/graph request may have ~45 minutes left to run. Two asks arriving 2 seconds apart must be served by capacity that is idle except for ask, and ask parallelism must scale with RAM and dev-agent count — without pretending enterprise scale today. The target is an initial safe envelope for the current box, explicitly not 75 simultaneous requests.

A dedicated endpoint alone cannot deliver that, and this is the sharpest finding in the lane. Ask admission is serialized in-process: SearchService.mjs:176 → buildChatModel → chatRequestQueue.enqueue (buildChatModel.mjs:160) → the process-wide sharedLocalChatRequestQueue, whose class is documented as "a single-lane request scheduler that runs async tasks one at a time" with no concurrency option. Two asks 2 s apart serialize regardless of endpoint or replica count — at the measured ~43 s each, the second returns near ~86 s. A pool provisioned without addressing admission yields idle replicas and a queued second ask, and the symptom reads as "the model is slow". So the constraint is admission, not capacity, and #17000 owns both together.

• Bind the shipped seam to a dedicated endpoint — the 3-local-model shape the config comment already describes (embed + summary + ask, each on its own port). Do not add config that exists.
• google/gemma-4-26b-a4b remains the full-power model for summaries and graph work. gemma-4-31b-it is excluded by operator decision.
• Candidate ask models are chosen by measurement, not preference. Gemma 4 ships five sizes — verified, not recalled: E2B (5.1B total / 2.3B active), E4B (8B), 12B dense, 26B A4B (26B / 4B active MoE), 31B dense; 256K context, so a ~12k prompt is nowhere near a limit. The family is not a constraint — non-Gemma candidates are in scope and should come from research rather than habit.
• The task being selected for is faithful summarisation of retrieved context — not algorithm or code solving. This is the load-bearing framing: a model that reasons well about code but paraphrases its sources loosely is the wrong choice here, and a leaderboard that rewards the former will mis-rank for us. It also makes abstention part of fidelity rather than a separate courtesy: declining when the context lacks the answer is faithful, and inventing a plausible one is not.
• The selection gate this parent decides against, two-stage (@neo-gpt-emmy relaying operator clarification, recorded here because the parent records the model decision while #17001 measures):
  • First pass — screening: max < 60 s at n ≥ 5. A candidate whose worst screening run exceeds the envelope is out; no averaging past a spike.
  • Final two — confirmation: p95 < 60 s at n ≥ 20. p95 is only claimed where the sample can carry it, which is why it is not the screening statistic.
  • Both at the default limit: 5, warm, real-tool end to end, including both requests of an overlap-2 round, with zero timeouts or degraded answers and incumbent-noninferior answer quality. Cold load is measured separately and never folded into the gate.
  • Block randomisation across rounds — interleaved single → overlap-2 → single → overlap-4, not sequential blocks, so drift and warming do not align with candidate order.
  • Explicit 80K and 120K long-guide / source stress rungs. Advertised 256K context does not prove a 120K prompt meets an interactive SLO.
  • If the host is swapping, selection is mechanically withheld rather than rationalised — the tripwire that keeps @neo-gpt-sol's allocation lane from being silently absorbed into a model verdict.
• Deliberately small candidate set, and the screen is ordered (operator clarification, not model preference): the full screen prioritises Gemma 4 12B and Qwen3.5 9B beside the incumbent Gemma 4 26B A4B. Qwen3.6 35B A3B is admitted to the full screen only after one representative warm sub-60 s smoke run — it is a conditional entrant, not the primary challenger. Qwen3.5 4B is a quality floor only, never assumed production-grade. Vendor benchmark tables are candidate-selection evidence, not our ask/RAG result.
• Quality is a first-class axis, scored explicitly. Asked about a concept absent from the corpus, the current model answered "I do not have enough information" and named what it did find. A faster model that fabricates instead of abstaining is a regression, and latency is trivially measurable while abstention-honesty is not — so it gets scored or it gets lost.
• A context bound with truncation stated in the answer — not a smaller limit default.
• Isolation verifiable from outside the process, by enumeration rather than assertion.

Lanes and dependency order

Three independently deliverable leaves. Peers self-select; nothing here is assigned.

#16999 and #17000 are parallel. #17001 wants both in place so its numbers are attributable.

The apparent cycle is broken deliberately: #17000 needs a model and #17001 produces the model evidence. So #17000 provisions the endpoint without choosing — it records the model it runs and that choice's memory consequence — #17001 measures candidates on it, and the model decision is recorded here on the parent, not inside either leaf.

Consumed from elsewhere, not owned here

• Host memory allocation. The box runs 128 GB with the container VM configured at 48 GiB while its containers use a fraction of that, and it swaps. That sizing decision is @neo-gpt-sol's lane. This lane consumes the outcome and must not endorse a number: Docker memory: values are ceilings rather than reservations, and a single docker stats snapshot cannot size a VM. A dedicated endpoint running a distinct model means a second resident chat model — askSynthesis.model's own comment warns of exactly that — which is an allocation consequence, stated here and decided there.
• Why 9.6 tok/sec. A 4-bit MLX model with 4B active parameters generating that slowly on a paging host is not evidence about model size. Selecting a smaller model before allocation resolves would confound the two and credit the wrong change.

Out of scope

• Embedding models — operator-set; nomic-embed-text is not a sufficient substitute. The embedding-side defect is a hardcoded 30-second deadline in our own code, a separate concern.
• A request-deadline authority. No caller deadline reaches the service today and no constant in ai/ names the client deadline. Plumbing one is a separate contract; no leaf here may claim to derive from it.
• The Ollama reproduction fixture — deleted. It mirrored the external plane's environment and must never become serving infrastructure: tuning a reproduction for our own latency destroys the property that made it a valid mirror.
• Corpus freshness — the corpus is stale (a query about work merged the same day returned no mention of it). Ingestion cadence is separate from synthesis capability.
• The external plane — this is our own tooling.

Avoided traps

• Rebuilding a shipped seam. This scope did exactly that for one round: it asserted no ask-only surface existed without grepping the config layer for one. An absence claim needs a positive control.
• Tuning the limit default and calling it fixed. Document count proxies token cost; the next large document re-crosses the cliff.
• Selecting a model before the host allocation is settled, which credits the wrong change.
• Trading quality for speed silently, because only one of the two is easy to measure.
• Reusing a test fixture as infrastructure — proposed in this lane's own design conversation and corrected: the isolation looked free because the fixture looked idle.

Refs #16706 Refs #12836