Context
ai/deploy/docker-compose.provider-lanes.yml requires every resource value fail-closed (:? substitution — CPUs, memory, slots, context, batch, ubatch) because "#17024 elects the fixed-envelope allocation; this profile consumes that result and never guesses it." Compute threads are the exception: nothing sets LLAMA_ARG_THREADS, so the engine guesses — from the wrong denominator.
At the pinned build, an unset LLAMA_ARG_THREADS resolves compute workers to the host's physical cores (common_cpu_get_num_math() — ~32 on the observed 32c/64t EPYC), and the HTTP pool separately defaults to max(n_parallel + 4, hardware_concurrency − 1) (63 there). Container CPU quotas (deploy.resources.limits.cpus) change neither answer — only cpusets would. The observed plane's 98 threads therefore decompose as ~32 compute + 63 HTTP + service threads: ~5.3× compute oversubscription inside a 6-cpu quota, plus a needlessly host-sized HTTP pool. (Corrected 2026-08-14 per Euclid's source-cited peer review — the original body over-read the process listing as "~64 compute / ~10×"; engine-source receipts in the correction comment.) Measured consequences from the plane's own ledger:
- tiny health-canary embeds averaged 12.8 s; "successful" canaries took 205–246 s — minutes for ~10-token inputs;
- the lane held ~591% of its 600% quota around the clock while storing almost nothing — compute spin-workers at ggml graph barriers convert quota into scheduler/cache thrash that reads as utilization (the 63 HTTP workers are a separate blocking pool and are not claimed as spinners; per-thread CPU attribution was not captured, so the clean before/after throughput is the falsifier for how much was overhead);
- every downstream instrument (canary latencies, probe outcomes, wall-time tables) was contaminated, which mis-aimed two prior root-cause analyses at capacity and at model size (see epic #17072).
The deployment-side fork has pinned threads as a mitigation; the canonical template still ships the defect to every future deployment.
The Problem
A lane whose thread counts derive from host hardware violates the template's own never-guess contract on the values that govern whether the CPU allocation does work or does overhead. Oversubscribed compute spinners are not a tuning nit: the lane presented as saturated around the clock while storing almost nothing — the exact "frozen cores" failure the lane split exists to prevent. How much of the burn was pure overhead is settled by the acceptance falsifier (clean before/after throughput and return-to-idle), not asserted from the process listing.
Architectural Reality
LLAMA_ARG_THREADS / LLAMA_ARG_THREADS_HTTP exist as env-mapped args at the pinned engine build (server-b10380).
- The election (#17024) already elects the lane's CPU allocation; threads must derive from that result via the same fail-closed substitution style as every sibling value.
ai/scripts/diagnostics/providerLaneComposition.mjs already direct-checks service env and pushes bounded error codes into the receipt's existing errors field (precedent: slots-endpoint-disabled) — the mechanical guard lands there with no receipt-shape change, so durable receipts and the election runner are unaffected.
- The Ollama chat lane has the same derivation class engine-side with no env to pin it — a documented gap, out of scope here (its heavy producers are being shaped in #17046).
The Fix (shape)
- Canonical lane env:
LLAMA_ARG_THREADS: ${NEO_PROVIDER_LANE_EMBEDDING_THREADS:?…} (fail-closed like every sibling) plus a pinned LLAMA_ARG_THREADS_HTTP, so /health stays answerable under full compute load.
- Analyzer rules (bounded error codes, existing
errors channel): embedding lane missing/non-integer LLAMA_ARG_THREADS → error; threads exceeding ceil(lane cpus) → error; missing LLAMA_ARG_THREADS_HTTP → error.
- A lane comment documenting the quota-vs-cpuset engine behavior, so the next compose fork keeps the pin instead of rediscovering the incident.
Acceptance Criteria
Out of Scope
- The healthcheck payload defect in the same service block (#17063 — sibling, separate leaf).
- Runtime live-shape verification against the elected envelope (#17069).
- Ollama engine-side thread control (no env exists; tracked as a documented gap).
Avoided Traps
- "The quota already limits CPU." It limits time, not thread count; oversubscribed spin workers burn the whole quota on synchronization, and the waste presents as work.
- "Use nproc inside the container."
nproc reads host affinity; under a quota (no cpuset) it still answers 64.
- "Fix the forks." The template is the generator; every unfixed render seeds the next incident.
Contract Ledger (backfilled 2026-08-14 per cross-family review RA-1 on PR #17074)
| Surface |
Producer |
Consumer |
Derivation |
Refusal posture |
NEO_PROVIDER_LANE_EMBEDDING_CPUS (existing elected input, now dual-consumed) |
#17024 election → closed deploymentInputs env export |
compose deploy.resources.limits.cpus and LLAMA_ARG_THREADS |
compute threads ≡ the elected CPU allocation, byte-identical (single authority; integer by the election contract) |
render fails closed without it; analyzer rejects threads !== cpuCores (embedding-threads-cpu-mismatch) and non-integer values (embedding-threads-unpinned) — a fractional allocation fails the integer gate and forces an explicit decision |
LLAMA_ARG_THREADS_HTTP (template constant 4) |
canonical template |
llama-server base HTTP pool (the engine may add dynamic workers under load; this is not a total cap) |
fixed small base pool; deliberately not elected — no per-host variance in need |
analyzer rejects missing (embedding-http-threads-unpinned) or > EMBEDDING_HTTP_THREADS_MAX (embedding-http-threads-oversized) — the host-derived hw−1 default cannot pass |
No new deployment input exists: the election actor's --env-file /dev/null + closed deploymentInputs export carries this contract unchanged, proven by the closed-input render arm in the composition spec.
Related
- #17063 — sibling defect in the same service block
- #17069 — runtime shape verification (consumes this pin as part of the envelope)
- #17024 — the election whose result the pin must consume
- Epic #17072 — incident family + operational doctrine
Live latest-open sweep: checked latest 20 open issues + recent A2A claims at creation time; no equivalent found (closest: #17063, deliberately separate leaf).
Context
ai/deploy/docker-compose.provider-lanes.ymlrequires every resource value fail-closed (:?substitution — CPUs, memory, slots, context, batch, ubatch) because "#17024 elects the fixed-envelope allocation; this profile consumes that result and never guesses it." Compute threads are the exception: nothing setsLLAMA_ARG_THREADS, so the engine guesses — from the wrong denominator.At the pinned build, an unset
LLAMA_ARG_THREADSresolves compute workers to the host's physical cores (common_cpu_get_num_math()— ~32 on the observed 32c/64t EPYC), and the HTTP pool separately defaults tomax(n_parallel + 4, hardware_concurrency − 1)(63 there). Container CPU quotas (deploy.resources.limits.cpus) change neither answer — only cpusets would. The observed plane's 98 threads therefore decompose as ~32 compute + 63 HTTP + service threads: ~5.3× compute oversubscription inside a 6-cpu quota, plus a needlessly host-sized HTTP pool. (Corrected 2026-08-14 per Euclid's source-cited peer review — the original body over-read the process listing as "~64 compute / ~10×"; engine-source receipts in the correction comment.) Measured consequences from the plane's own ledger:The deployment-side fork has pinned threads as a mitigation; the canonical template still ships the defect to every future deployment.
The Problem
A lane whose thread counts derive from host hardware violates the template's own never-guess contract on the values that govern whether the CPU allocation does work or does overhead. Oversubscribed compute spinners are not a tuning nit: the lane presented as saturated around the clock while storing almost nothing — the exact "frozen cores" failure the lane split exists to prevent. How much of the burn was pure overhead is settled by the acceptance falsifier (clean before/after throughput and return-to-idle), not asserted from the process listing.
Architectural Reality
LLAMA_ARG_THREADS/LLAMA_ARG_THREADS_HTTPexist as env-mapped args at the pinned engine build (server-b10380).ai/scripts/diagnostics/providerLaneComposition.mjsalready direct-checks service env and pushes bounded error codes into the receipt's existingerrorsfield (precedent:slots-endpoint-disabled) — the mechanical guard lands there with no receipt-shape change, so durable receipts and the election runner are unaffected.The Fix (shape)
LLAMA_ARG_THREADS: ${NEO_PROVIDER_LANE_EMBEDDING_THREADS:?…}(fail-closed like every sibling) plus a pinnedLLAMA_ARG_THREADS_HTTP, so/healthstays answerable under full compute load.errorschannel): embedding lane missing/non-integerLLAMA_ARG_THREADS→ error; threads exceedingceil(lane cpus)→ error; missingLLAMA_ARG_THREADS_HTTP→ error.Acceptance Criteria
analyzeProviderLaneCompositionrejects a composition whose embedding service lacksLLAMA_ARG_THREADSor whose value exceedsceil(lane cpus), with distinguishing error codes (unit arms).LLAMA_ARG_THREADS_HTTPis pinned and asserted.Out of Scope
Avoided Traps
nprocreads host affinity; under a quota (no cpuset) it still answers 64.Contract Ledger (backfilled 2026-08-14 per cross-family review RA-1 on PR #17074)
NEO_PROVIDER_LANE_EMBEDDING_CPUS(existing elected input, now dual-consumed)deploymentInputsenv exportdeploy.resources.limits.cpusandLLAMA_ARG_THREADSthreads !== cpuCores(embedding-threads-cpu-mismatch) and non-integer values (embedding-threads-unpinned) — a fractional allocation fails the integer gate and forces an explicit decisionLLAMA_ARG_THREADS_HTTP(template constant4)embedding-http-threads-unpinned) or> EMBEDDING_HTTP_THREADS_MAX(embedding-http-threads-oversized) — the host-derivedhw−1default cannot passNo new deployment input exists: the election actor's
--env-file /dev/null+ closeddeploymentInputsexport carries this contract unchanged, proven by the closed-input render arm in the composition spec.Related
Live latest-open sweep: checked latest 20 open issues + recent A2A claims at creation time; no equivalent found (closest: #17063, deliberately separate leaf).