Refs #16998
Problem
Why separation is load-bearing (operator, relayed via @neo-gpt-emmy): the reason is scalability, not tidiness. A full-power Dream/graph request may have ~45 minutes left to run. Two asks arriving 2 seconds apart must be served by capacity that is idle except for ask, and ask parallelism must scale with RAM and dev-agent count — without pretending enterprise scale today.
The config seam for this already ships (#12836), and this ticket must not rebuild it. Verified at dev 271bb132d85b8219d9565c201ea198f39d99029a:
| shipped substrate |
where |
askSynthesis node — provider, model, apiKey, baseUrl, timeoutMs, timeoutMsRemote, maxCallsPerMinute |
ai/mcp/server/knowledge-base/configBase.mjs:206 |
| synthesis model built from the dedicated block, source comment says "NOT the global" |
ai/services/knowledge-base/SearchService.mjs:155 |
NEO_KB_ASK_PROVIDER/MODEL/API_KEY/BASE_URL pass-through |
ai/deploy/docker-compose.yml:142, docker-compose.dev.yml:198 |
| all eight leaves parity-locked |
ai/scripts/lint/config-leaf-parity.json:499 |
| missing-leaf guard → degraded-references envelope |
ai/services/knowledge-base/helpers/askSynthesisGuard.mjs |
askSynthesis.baseUrl defaults to null, resolving to the provider's default host — the same endpoint serving everything else. The seam is present and unbound, and its own comment names the intended shape: "set when the ask model runs on its OWN endpoint (3-local-model setup: embed + summary + ask each on its own port)."
A dedicated endpoint alone cannot satisfy the capacity requirement
This is the finding that shapes the ticket, and it is why @neo-gpt-emmy's L3 witness is the right discriminator — it falsifies the deployment-only fix. Traced at the same SHA:
SearchService.mjs:3,176 import {buildChatModel} … this.model = buildChatModel({…})
buildChatModel.mjs:160 return chatRequestQueue.enqueue(async () => { … }) // openAiCompatible = ask's default provider
buildChatModel.mjs:110 chatRequestQueue = sharedLocalChatRequestQueue // process-wide default
InteractiveBatchQueue.mjs:2 "A single-lane request scheduler that runs async tasks one at a time"
InteractiveBatchQueue.mjs:38 #active — "guards against concurrent drains"; one drain loop, no concurrency optionAsk admission is serialized in-process. Two asks 2 seconds apart are serialized regardless of which endpoint or how many replicas they target: at the measured ~43 s per ask, the second returns near ~86 s. Provisioning a pool without addressing admission yields idle replicas and a queued second ask — and the symptom would read as "the model is slow", not "the queue admitted one".
So the constraint is admission, not capacity, and both must be delivered together.
Priority does not rescue this, and it is worth saying because it looks like it should. Ask already dispatches at interactive (SearchService.mjs:553), and the queue prefers any waiting interactive task over batch. But the class is explicit that tasks are non-preemptible — an already-running task always finishes — so the lane preference only decides who waits next. It does nothing for ask-vs-ask at equal priority, and nothing at all across processes. Three distinct mechanisms, and only the third is already handled:
| collision |
mechanism |
status |
| ask vs. summarisation/graph in another process |
no shared queue; they meet at the serving endpoint |
this ticket — endpoint isolation |
| ask vs. ask, same process, same priority |
FIFO admission, one at a time |
this ticket — parallel capacity |
| ask vs. queued batch, same process |
interactive preferred over batch |
already handled — no work |
The witness verdict is decomposable from persisted data, not inferred from a total (@neo-opus-ada, verified at source). queue_wait_ms is already computed and persisted for every ask — providerActivityLedger.mjs:147, schema :120-121, 'neo-queued' default at :258, read-back :416. And the split the witness needs is one shipped SELECT, not a build: :357-360 returns avg/max_queue_wait_ms and avg/max_execution_ms in the same aggregate row, already grouped by priority; :446-449 is the live waiting vs executing form of the same split.
So the two-asks-2s-apart run does not rest on a stopwatch: if the second ask's queue_wait_ms ≈ the first ask's duration, that is the serialized-admission signature, in a row, and a pass must show that wait collapse to near zero. No new instrumentation is needed to judge this ticket's own witness, and the judging cannot perturb what it measures.
⚠️ The witness is valid on our plane only, and this bound is load-bearing (@neo-opus-ada's retraction of her own historical-query aside). accfdb0c1a — native admission enforced and then described as absent (#16943) — is not an ancestor of the external plane's pinned revision. Verified here rather than carried from a peer's ancestry check, after a false deployed-SHA was found circulating in shared context:
accfdb0c1a 2026-08-11 17:51 the fix
3f9f8343a8 2026-08-10 16:25 the pinned revision — a day EARLIER
git merge-base --is-ancestor accfdb0c1a 3f9f8343a8 -> false
Pre-fix rows there report a real queue as absent, so a queue_wait_ms read against that plane encodes the fixed defect rather than measuring admission. Do not run this witness against the external plane until it carries accfdb0c1a.
The path that satisfies the capacity requirement without rebuilding routing
InteractiveBatchQueue is documented as a "reusable provider-layer primitive… deliberately a plain class, not a Neo singleton" (:20). Giving it an optional parallel capacity that defaults to 1 leaves every existing consumer byte-identical, including memory-core's embedding path.
- Capacity N belongs as a leaf in the existing
askSynthesis node — no second config node.
SearchService must own the read and the injection, and this ticket therefore DOES change it (@neo-gpt-emmy's use-site correction, which also corrects her own earlier "without changing SearchService" and my encoding of it). ADR 0019 makes the alternatives unlawful, not merely inelegant:
- Threading the number —
buildChatModel({…, maxParallel}) — is B5: "passing AiConfig values into other consumers' configs", whose sanctioned form is that the consumer reads it itself.
- Having
buildChatModel read AiConfig.askSynthesis is C1 ⛔ zero-tolerance: Neo/AiConfig imports belong only in thread-entrypoints, and a provider-layer module is not one.
- So
buildChatModel cannot lawfully discover the capacity at all. SearchService already reads aiConfig.askSynthesis lawfully (:160), so it reads the capacity leaf at its use site, constructs the queue, and injects the constructed collaborator. Injecting an object is not threading a config value — that is the distinction that keeps this inside B5.
- ⚠️ The injection parameter is labelled a test seam (
buildChatModel.mjs:95). Promoting a test seam to a production injection point is a deliberate decision to argue in the PR, not a free win — if it is the wrong channel, the honest alternative is a named production capacity parameter, not quiet reuse.
Acceptance criteria
Post-Merge Validation
Residual-Owner: #16998
Consumed from elsewhere, not owned here
- Host memory allocation — @neo-gpt-sol's lane. Parallel ask capacity and a possible second resident chat model are allocation consequences; this leaf states them and must not endorse a number. Docker
memory: values are ceilings, not reservations.
- Which model runs there — #17001's measured output, including its concurrency cohort.
Out of scope
- The context budget — #16999. Note the two interact: parallel asks multiply peak context memory, so the bound lands first or the witness measures an unbounded prompt.
- Embedding models and their endpoint — operator-set, separate concern.
- Enterprise scale. The target is an initial safe envelope sized to today's RAM and dev-agent count, explicitly not 75 simultaneous requests.
Authored by @neo-opus-vega
Deltas
ai/deploy/docker-compose*.yml + deployment env — provision and bind the dedicated ask endpoint/pool.
ai/provider/InteractiveBatchQueue.mjs — optional parallel capacity, default 1.
ai/mcp/server/knowledge-base/configBase.mjs — the capacity leaf inside the existing askSynthesis node.
ai/services/knowledge-base/SearchService.mjs — reads the capacity leaf at its use site, constructs the ask-owned queue, injects it. This change is required, not optional: ADR 0019 B5 forbids threading the value and C1 forbids buildChatModel reading it.
No new config node, and no duplicative routing in buildChatModel. If the test-seam parameter is not the right production channel, a named capacity parameter replaces it — argued in the PR, not assumed.
Evidence: L3 required — endpoint exclusivity, external attribution and the two-asks-2s-apart discriminator are runtime claims no spec arm can discharge. L2 arms still required for the queue capacity default (byte-identical at 1) and the config leaf.
Test Evidence
To be recorded on the implementing PR.
Refs #16998
Problem
Why separation is load-bearing (operator, relayed via @neo-gpt-emmy): the reason is scalability, not tidiness. A full-power Dream/graph request may have ~45 minutes left to run. Two asks arriving 2 seconds apart must be served by capacity that is idle except for ask, and ask parallelism must scale with RAM and dev-agent count — without pretending enterprise scale today.
The config seam for this already ships (#12836), and this ticket must not rebuild it. Verified at dev
271bb132d85b8219d9565c201ea198f39d99029a:askSynthesisnode —provider,model,apiKey,baseUrl,timeoutMs,timeoutMsRemote,maxCallsPerMinuteai/mcp/server/knowledge-base/configBase.mjs:206ai/services/knowledge-base/SearchService.mjs:155NEO_KB_ASK_PROVIDER/MODEL/API_KEY/BASE_URLpass-throughai/deploy/docker-compose.yml:142,docker-compose.dev.yml:198ai/scripts/lint/config-leaf-parity.json:499ai/services/knowledge-base/helpers/askSynthesisGuard.mjsaskSynthesis.baseUrldefaults tonull, resolving to the provider's default host — the same endpoint serving everything else. The seam is present and unbound, and its own comment names the intended shape: "set when the ask model runs on its OWN endpoint (3-local-model setup: embed + summary + ask each on its own port)."A dedicated endpoint alone cannot satisfy the capacity requirement
This is the finding that shapes the ticket, and it is why @neo-gpt-emmy's L3 witness is the right discriminator — it falsifies the deployment-only fix. Traced at the same SHA:
SearchService.mjs:3,176 import {buildChatModel} … this.model = buildChatModel({…}) buildChatModel.mjs:160 return chatRequestQueue.enqueue(async () => { … }) // openAiCompatible = ask's default provider buildChatModel.mjs:110 chatRequestQueue = sharedLocalChatRequestQueue // process-wide default InteractiveBatchQueue.mjs:2 "A single-lane request scheduler that runs async tasks one at a time" InteractiveBatchQueue.mjs:38 #active — "guards against concurrent drains"; one drain loop, no concurrency optionAsk admission is serialized in-process. Two asks 2 seconds apart are serialized regardless of which endpoint or how many replicas they target: at the measured ~43 s per ask, the second returns near ~86 s. Provisioning a pool without addressing admission yields idle replicas and a queued second ask — and the symptom would read as "the model is slow", not "the queue admitted one".
So the constraint is admission, not capacity, and both must be delivered together.
Priority does not rescue this, and it is worth saying because it looks like it should. Ask already dispatches at
interactive(SearchService.mjs:553), and the queue prefers any waitinginteractivetask overbatch. But the class is explicit that tasks are non-preemptible — an already-running task always finishes — so the lane preference only decides who waits next. It does nothing for ask-vs-ask at equal priority, and nothing at all across processes. Three distinct mechanisms, and only the third is already handled:interactivepreferred overbatchThe witness verdict is decomposable from persisted data, not inferred from a total (@neo-opus-ada, verified at source).
queue_wait_msis already computed and persisted for every ask —providerActivityLedger.mjs:147, schema:120-121,'neo-queued'default at:258, read-back:416. And the split the witness needs is one shipped SELECT, not a build::357-360returnsavg/max_queue_wait_msandavg/max_execution_msin the same aggregate row, already grouped bypriority;:446-449is the livewaitingvsexecutingform of the same split.So the two-asks-2s-apart run does not rest on a stopwatch: if the second ask's
queue_wait_ms≈ the first ask's duration, that is the serialized-admission signature, in a row, and a pass must show that wait collapse to near zero. No new instrumentation is needed to judge this ticket's own witness, and the judging cannot perturb what it measures.⚠️ The witness is valid on our plane only, and this bound is load-bearing (@neo-opus-ada's retraction of her own historical-query aside).
accfdb0c1a— native admission enforced and then described as absent (#16943) — is not an ancestor of the external plane's pinned revision. Verified here rather than carried from a peer's ancestry check, after a false deployed-SHA was found circulating in shared context:Pre-fix rows there report a real queue as absent, so a
queue_wait_msread against that plane encodes the fixed defect rather than measuring admission. Do not run this witness against the external plane until it carriesaccfdb0c1a.The path that satisfies the capacity requirement without rebuilding routing
InteractiveBatchQueueis documented as a "reusable provider-layer primitive… deliberately a plain class, not a Neo singleton" (:20). Giving it an optional parallel capacity that defaults to 1 leaves every existing consumer byte-identical, including memory-core's embedding path.askSynthesisnode — no second config node.SearchServicemust own the read and the injection, and this ticket therefore DOES change it (@neo-gpt-emmy's use-site correction, which also corrects her own earlier "without changingSearchService" and my encoding of it). ADR 0019 makes the alternatives unlawful, not merely inelegant:buildChatModel({…, maxParallel})— is B5: "passingAiConfigvalues into other consumers' configs", whose sanctioned form is that the consumer reads it itself.buildChatModelreadAiConfig.askSynthesisis C1 ⛔ zero-tolerance: Neo/AiConfigimports belong only in thread-entrypoints, and a provider-layer module is not one.buildChatModelcannot lawfully discover the capacity at all.SearchServicealready readsaiConfig.askSynthesislawfully (:160), so it reads the capacity leaf at its use site, constructs the queue, and injects the constructed collaborator. Injecting an object is not threading a config value — that is the distinction that keeps this inside B5.buildChatModel.mjs:95). Promoting a test seam to a production injection point is a deliberate decision to argue in the PR, not a free win — if it is the wrong channel, the honest alternative is a named production capacity parameter, not quiet reuse.Acceptance criteria
NEO_KB_ASK_BASE_URLseam. No second ask-provider config node, and nobuildChatModelchange that merely recreates existing routing.askSynthesisnode, read at the use site bySearchService— which constructs the ask queue and injects it. The capacity value is never threaded intobuildChatModel(B5) andbuildChatModelnever importsAiConfig(C1 ⛔). No re-derivation, no env re-read, no hidden default, no defensive?..askSynthesis.model's own warning). This leaf does not select the model; that is #17001's output.Post-Merge Validation
queue_wait_msof each ask, not on end-to-end totals: the second ask's wait must collapse to near zero. A wait ≈ the first ask's duration is a failure, and specifically the serialized-admission signature rather than a slow model.Residual-Owner: #16998
Consumed from elsewhere, not owned here
memory:values are ceilings, not reservations.Out of scope
Authored by @neo-opus-vega
Deltas
ai/deploy/docker-compose*.yml+ deployment env — provision and bind the dedicated ask endpoint/pool.ai/provider/InteractiveBatchQueue.mjs— optional parallel capacity, default 1.ai/mcp/server/knowledge-base/configBase.mjs— the capacity leaf inside the existingaskSynthesisnode.ai/services/knowledge-base/SearchService.mjs— reads the capacity leaf at its use site, constructs the ask-owned queue, injects it. This change is required, not optional: ADR 0019 B5 forbids threading the value and C1 forbidsbuildChatModelreading it. No new config node, and no duplicative routing inbuildChatModel. If the test-seam parameter is not the right production channel, a named capacity parameter replaces it — argued in the PR, not assumed.Evidence: L3 required — endpoint exclusivity, external attribution and the two-asks-2s-apart discriminator are runtime claims no spec arm can discharge. L2 arms still required for the queue capacity default (byte-identical at 1) and the config leaf.
Test Evidence
To be recorded on the implementing PR.