Ask synthesis dispatches through a single-lane admission queue, so two asks arriving seconds apart are served one after the other regardless of how much idle capacity the serving endpoint has. Traced at 271bb132d8:
SearchService.mjs:176 this.model = buildChatModel({…})
buildChatModel.mjs:160 return chatRequestQueue.enqueue(…) // openAiCompatible = ask's default provider
buildChatModel.mjs:110 chatRequestQueue = sharedLocalChatRequestQueue // process-wide default
InteractiveBatchQueue.mjs:2 "A single-lane request scheduler that runs async tasks one at a time"
At the measured ~43 s per ask, the second returns near ~86 s.
This is why provisioning capacity alone cannot fix it. A dedicated ask endpoint, or a replica pool, yields idle replicas behind a queued second ask — and the symptom reads as "the model is slow" rather than "the queue admitted one". The constraint is admission, and it is invisible to every latency measurement that does not separate queue wait from execution time.
Carved out of #17000 after @neo-gpt's review of PR #17007: that PR delivers admission only, and closing #17000 with it would assign #17000's load-bearing remainder — the dedicated endpoint, exclusivity enumeration, external attribution, model decision and the L3 witness — back to a ticket declared resolved. This ticket is the part a PR can fully close; #17000 keeps the rest.
Acceptance criteria
InteractiveBatchQueue accepts an optional parallel capacity, defaulting to 1 — byte-identical to the previous single-lane scheduler, so every existing consumer is unchanged and evidenced as such.
An unusable capacity (0, negative, fractional, non-numeric) is refused at construction, not at admission: a stalled lane is indistinguishable from a hung provider on the caller's side.
Ask parallelism is operator-adjustable through a leaf in the existingaskSynthesis node, read at the use site — no second config node, no threaded value (ADR 0019 B5), no AiConfig import in the provider layer (C1).
The leaf is reachable from a deployment — carried through the canonical Compose profiles. A control with an env binding and no compose surface is not operator-adjustable.
Ask receives its own queue instance rather than a raised capacity on the shared one, so parallelism cannot leak to other consumers in the process.
Spec: the queue is HANDED to the provider layer, not merely constructible. Removing the injection must fail an arm — a helper-capacity assertion proves the helper, not the delivery.
Spec: slot-filling admits two same-tick enqueues when capacity allows; a freed slot pulls the next waiting task; a throwing task releases its slot.
Contract Ledger
Target surface
Source of authority
Behaviour
Failure / fallback
Evidence
InteractiveBatchQueue
this ticket
admits up to capacity; interactive still preferred when a slot frees
capacity 1 reproduces the single-lane scheduler exactly
new arms + the original arms as the default-capacity control
construction guard
this ticket
integer >= 1 required
anything else throws at construction
explicit arm with a valid-capacity control
askSynthesis.maxParallel
KB AiConfig, read at the use site
sizes the ask-owned queue
no builder-local fallback; a missing leaf lands on the primitive's own default
buildAskRequestQueue arms
Compose profiles
this ticket
the leaf is settable on a deployment
absent env → the declared default
grep with a sibling-leaf positive control
provider composition
this ticket
the ask-owned queue reaches buildChatModel
removing the injection fails an arm
composition arm, mutation-verified
Out of scope
The dedicated serving endpoint, its exclusivity enumeration, external attribution, the model decision and the two-asks-2s-apart L3 witness — all remain on #17000, which needs host provisioning this ticket does not.
The context budget (#16999) and model selection (#17001).
Related
#17000 — parent lane; keeps the endpoint work this ticket deliberately excludes
#16998 — the epic
PR #17007 — the implementation, reviewed by @neo-gpt
Live latest-open sweep at creation: #17006 (21:57Z), #17004 (21:56Z), #17001 / #17000 / #16999 (19:43Z) — no equivalent. Duplicate sweep on "admission queue capacity ask parallel" and InteractiveBatchQueue in:title: only #17000 / #16998, which this carves from. A2A: no lane-claim on this surface; #17000 is assigned to me.
tobiu referenced in commit 030aeec - "feat(provider): bound ask admission so a dedicated endpoint can actually be used (#17000) (#17007) on Aug 12, 2026, 10:11 AM
Problem
Ask synthesis dispatches through a single-lane admission queue, so two asks arriving seconds apart are served one after the other regardless of how much idle capacity the serving endpoint has. Traced at
271bb132d8:SearchService.mjs:176 this.model = buildChatModel({…}) buildChatModel.mjs:160 return chatRequestQueue.enqueue(…) // openAiCompatible = ask's default provider buildChatModel.mjs:110 chatRequestQueue = sharedLocalChatRequestQueue // process-wide default InteractiveBatchQueue.mjs:2 "A single-lane request scheduler that runs async tasks one at a time"At the measured ~43 s per ask, the second returns near ~86 s.
This is why provisioning capacity alone cannot fix it. A dedicated ask endpoint, or a replica pool, yields idle replicas behind a queued second ask — and the symptom reads as "the model is slow" rather than "the queue admitted one". The constraint is admission, and it is invisible to every latency measurement that does not separate queue wait from execution time.
Carved out of #17000 after @neo-gpt's review of PR #17007: that PR delivers admission only, and closing #17000 with it would assign #17000's load-bearing remainder — the dedicated endpoint, exclusivity enumeration, external attribution, model decision and the L3 witness — back to a ticket declared resolved. This ticket is the part a PR can fully close; #17000 keeps the rest.
Acceptance criteria
InteractiveBatchQueueaccepts an optional parallel capacity, defaulting to1— byte-identical to the previous single-lane scheduler, so every existing consumer is unchanged and evidenced as such.0, negative, fractional, non-numeric) is refused at construction, not at admission: a stalled lane is indistinguishable from a hung provider on the caller's side.askSynthesisnode, read at the use site — no second config node, no threaded value (ADR 0019 B5), noAiConfigimport in the provider layer (C1).Contract Ledger
InteractiveBatchQueuecapacity; interactive still preferred when a slot frees1reproduces the single-lane scheduler exactly>= 1requiredaskSynthesis.maxParallelAiConfig, read at the use sitebuildAskRequestQueuearmsbuildChatModelOut of scope
Related
Live latest-open sweep at creation: #17006 (21:57Z), #17004 (21:56Z), #17001 / #17000 / #16999 (19:43Z) — no equivalent. Duplicate sweep on "admission queue capacity ask parallel" and
InteractiveBatchQueue in:title: only #17000 / #16998, which this carves from. A2A: no lane-claim on this surface; #17000 is assigned to me.Retrieval Hint:
ask admission single-lane queue capacity maxParallel compose reachable injection witness InteractiveBatchQueueAuthored by @neo-opus-vega