The embedding dispatch loop awaits every request, so a lane declaring four parallel slots runs one

Context

localModels.embedding.parallel is the single-source declaration for how many embedding requests may be in flight — the leaf exists so one value applies whichever provider serves the lane (LM Studio, Ollama, llama.cpp). Nothing on the dispatch path honours it.

The Problem

TextEmbeddingService.mjs:1930 iterates provider requests sequentially:

for (let offset = 0; offset < texts.length; offset += chunkSize) {
    result = await this.#enqueueOpenAiCompatiblePost(chunk, {...})
}

One POST is awaited before the next is issued. Measured on a live external tenant deployment 2026-08-20 (read-only), the provider's slot log shows ~20 consecutive launch → release → launch transitions with never two tasks in flight, slots selected round-robin by LRU:

387.04.023  launch id 1  task 9776
387.04.528  release id 1  task 9776   113 tok
387.04.537  launch id 3  task 9778
387.07.412  release id 3  task 9778   640 tok

The four slots are a rotation, not concurrency. The provider queues — it is not refusing work, it is never given any to overlap.

Two consequences on that plane:

• A single long input occupies the entire lane. One 13,725-token chunk consumed a whole 5-minute slice budget while three slots idled. Cost on this lane fits ∝ n^1.58 (≈400 tokens ≈ 1 s; 13,725 ≈ 268 s), so one long input is worth hundreds of short ones — and currently blocks every one of them.
• The capacity is demonstrably there. The same lane sustained ~19 embeddings/min on 100–1,400-token inputs within the same window.

The Architectural Reality

Directly above the loop:

slotHeadroomWidth = Number.isInteger(embeddingParallel) && embeddingParallel > 1 ?
    embeddingParallel - 1 :
    configuredChunkSize,
chunkSize = Math.min(configuredChunkSize, slotHeadroomWidth),

This is the only consumer of the declared parallelism on this path, and it uses it to compute a request width, not a concurrency. Its stated purpose is to leave one slot free so other traffic can interleave, and that purpose is correct — see the 2026-08-21 amendment below. An earlier revision of this paragraph claimed a client cannot reserve a slot by sending fewer inputs; the provider's capacity unit is a TASK, so it can. What is wrong is the UNIT the clamp is spent in: headroom expressed as a request width consumes the same budget concurrency needs, so a lane declaring four slots offered one request at a time.

The reserved fraction is also accidental: parallel − 1 reserves 25% at four slots, 50% at two, 6% at sixteen. And the clamp only ever fires downward — with parallel <= 1 it falls through unclamped — so its entire effect is to reduce capacity on the multi-slot lanes it was written for.

Separately, the sweep's own concurrency is not a settable leaf at all (#17158), so a deployment cannot raise in-flight width from the outside either.

The Fix

Bound in-flight requests by localModels.embedding.parallel and let the provider schedule. Delete the slotHeadroomWidth clamp: request width comes from the durability contract, concurrency comes from the declared parallelism, and the two stop being conflated. If interleaving headroom for interactive probes is a real requirement, it belongs where that traffic is admitted — not as an unstated capacity tax on batch work.

The existing per-chunk yield point and its forward-progress guarantee (at least one chunk lands per acquisition) must survive: concurrency changes how many requests are outstanding, never whether a reached checkpoint is durable.

⚠️ AMENDED 2026-08-20 — what "must survive" actually costs, verified before implementing

That sentence was one line and it is the whole difficulty. Five properties of this loop are stated in terms of a CONTIGUOUS PREFIX, and fan-out breaks the arithmetic they rest on. Read at TextEmbeddingService.mjs:1991-1998:

const completedTextCount = completedChunkCount * chunkSize,
      embeddings         = toOrderedEmbeddings(data, completedTextCount);

completedChunkCount * chunkSize is a count multiplied by a width. It identifies a span only because completions arrive in order from offset 0 — which the production comment states outright: "Completed chunks are full-width by the same boundary argument the yield error makes (only a batch's final chunk can be short…), so the expected count derives from what was SENT."

With N requests outstanding, the completed set can be non-contiguous. Chunk 0 and chunk 2 may succeed while chunk 1 fails. Then completedChunkCount = 2, completedTextCount = 2 × chunkSize, and data holds indices from chunks 0 and 2 — a hole inside the claimed span and entries beyond it.

The consequence is not a mis-binding; it is worse than that. toOrderedEmbeddings's positional-binding guard fails closed, so the catch swallows and the error "travels uncarried". So concurrency does not corrupt vectors — it silently disables work conservation, which is precisely the regression the carry was added to fix. Every fan-out failure re-purchases the whole completed set, and nothing reports it: the lane just runs slower forever.

The five prefix-dependent properties, each needing an explicit answer rather than inheritance:

So the implementation is a representation change, not a wrapper. The carry must become explicitly indexed — the set of completed spans, not a count — and the guard must distinguish "nothing landed" from "something landed non-contiguously", because those are different facts and only one of them is a reason to drop the carry.

Discovered by reading the carry arithmetic before writing code rather than after. Recorded here because an implementation that added concurrency and left count × width in place would pass a naive concurrency arm, ship a silent conservation regression, and present as a throughput fix.

Contract Ledger Matrix

AMENDED 2026-08-21 (second pass) on @neo-gpt's reopened P1. What each declared budget means, so the specs are anchored to the AC rather than to whichever number made an arm observable:

• parallel = 4 — batch work peaks at 3; batch plus a concurrent interactive post peaks at 4. This is the reservation, and it is the case the ticket's headroom contract is about.
• parallel = 5 — the exact-peak-4 fan-out arm. Budget 5 is required to observe a four-wide batch peak because of the reservation, so that arm measures the worker-pool mechanism, not the headroom rule.

AMENDED 2026-08-21 on @neo-gpt's P1 (PR #17433). The prior table's second row carried the claim his review falsified — "a client cannot reserve a server slot by sending fewer inputs" — and its first row measured concurrency in requests. Both are corrected below, and the four surfaces the correction introduced are added.

Decision Record impact

aligned-with ADR 0014 — the two-lane provider split is unchanged; this alters only how many requests the client keeps outstanding on the embedding lane.

Acceptance Criteria

• RED-PROOF, concurrency: concurrency is observed by OVERLAP against a recording provider stub, never by counting calls — a sequential loop makes the same number of calls. The exact peak is budget-dependent because of the interactive reservation, and an earlier revision of this AC said parallel = 4 yields at least four concurrent requests, which the reservation makes false: at parallel = 4 batch work peaks at 3, and batch plus a concurrent interactive post peaks at 4. The four-wide batch fan-out is observable at parallel = 5, and that arm measures the worker pool rather than the headroom rule. Both peaks are asserted exactly, never as a lower bound.
• SILENT ARM: with parallel = 1 (or unset), behaviour is byte-identical to today — one request outstanding, same ordering. Without this arm a change that always fans out passes the arm above.
• RED-PROOF, head-of-line: one slow input does not delay unrelated pending inputs. Asserted with a stub whose first request resolves last: the later inputs complete before it, and the assembled output stays in input order.
• The declared value is the only authority. slotHeadroomWidth is gone, and no site derives a width or a concurrency from parallel by arithmetic. A fixture asserts that changing parallel changes observed concurrency, so it is not a constant wearing a leaf's name.
• Forward progress is preserved under concurrency. A lease yield mid-flight still lands every completed request durably; a repeatedly-yielding acquisition never re-embeds a prefix it already completed. Asserted on the yield path, not only the happy path.
• RED-PROOF, non-contiguous conservation — the arm this ticket was missing. With four outstanding requests where an EARLY one fails and LATER ones succeed, the carried result still reports every completed span. Asserted against today's tree it must FAIL, and it must fail by carrying nothing rather than by mis-binding — the positional guard fails closed, so the defect is a silent loss of work conservation, not a wrong vector. An arm that only checks "no wrong vectors" passes on the broken shape.
• completedTextCount is no longer a product. No site derives a completed span from count × width; a fixture with a deliberate hole in the completed set proves the carry names the real spans. This is the representation change the ticket rests on, so it is asserted directly rather than implied by the arm above.
• "Nothing landed" and "something landed out of order" are distinguishable. The first is a reason to drop the carry; the second is not, and today they are the same code path. Asserted by observing the carry survive a non-contiguous completion.
• Failure attribution survives fan-out. A single failing request among four outstanding attributes its strikes to its own inputs only. Asserted with three concurrent successes and one failure.
• Ordering is contract, not accident: outputs map to inputs by index regardless of completion order, asserted with deliberately inverted completion.

Out of Scope

• The corpus chunk ceiling. 16,384 tokens per chunk stays — source-code chunks must remain semantically whole for retrieval.
• Provider memory or CPU allocation. This ticket makes the existing allocation reachable; it does not ask for more.
• Making the sweep's concurrency configurable — that is #17158, and it composes with this rather than being fixed by it.
• Retiring the compensating layers this unblocks. Deliberately deferred: which of them still fire is only decidable once the lane runs at its declared width, and that census is governed by the parent epic.

Avoided Traps

• Do not widen the request instead. A wider POST with a sequential loop changes little and would present as a fix that did nothing.
• Do not add a second concurrency authority. The parallelism leaf exists so one value spans providers; a new maxConcurrentEmbedRequests beside it recreates the drift this ticket removes.
• Do not read the idle slots as a provider limitation. The provider queues and idles when nothing arrives; the bound is client-side.
• Do not assert concurrency by call count. A sequential loop makes N calls too. The arm has to observe overlapping lifetimes or it proves nothing.

Related

Parent: #17411. Composes with #17158 (sweep concurrency not settable) and #16972 (batch retried at identical size). Outcome bar: D#17136.