ai/deploy/docker-compose.provider-lanes.yml is explicit about its own contract:
Every resource value is required because #17024 elects the fixed-envelope allocation; this profile consumes that result and never guesses it.
It enforces that with fail-closed substitution — ${NEO_PROVIDER_LANE_EMBEDDING_SLOTS:?embedding slots required}, …TOTAL_CONTEXT_TOKENS:?, …UBATCH_TOKENS:?, …EMBEDDING_CPUS:? — and ships a validator (ai/scripts/diagnostics/providerLaneComposition.mjs) plus an election runner (ProviderLaneElectionRunner) that benchmarks candidate slot counts against the actual hardware. A per-slot floor appears as NEO_PROVIDER_LANE_EMBEDDING_CONTEXT_TOKENS_PER_SLOT_REQUIRED (8192 in the test fixtures — a fixture value, not a shipped default; see the correction below).
That protection is defeated by copying the file.
An external plane was observed on 2026-08-13 running a compose whose embedding-model block is byte-identical to the canonical one — same image digest, same entrypoint, same healthcheck — except that every :?-required value had been replaced with a literal:
Correction (2026-08-14, replacing this section's original worked example — the original is preserved in the correction comment): the authoritative floor is safeProcessingLimitTokens (28,672, ai/configBase.mjs) — Neo deliberately sends single embedding inputs up to that size — and against that band the observed plane's 32,768 tokens per slot is correct (the smallest power of two holding a safe-band input; #17070 files the floor linkage). The engine reported n_slots = 4, n_ctx_slot = 32768. The plane's slow requests (a 9,144-token embed measured ~11 minutes) were later traced to engine thread oversubscription inside the container CPU quota plus a heavyweight healthcheck (#17063) — pathology-contaminated numbers, not evidence of a wrong envelope.
The defect this ticket keeps is narrower and fully stands: the values were substituted as literals, so nothing verified them. A deployment sized below the safe band would have run exactly as silently — in the direction that corrupts data instead of merely running slow.
The Problem
The fail-closed :? guard protects exactly one thing: rendering this file without an elected envelope. It cannot protect against the far more likely operational path — a deployment that vendors its own compose and substitutes literals, whether by copy-paste, by templating, or because the deployment topology needs services the canonical overlay does not include.
Once that happens:
The election (#17024) never runs, so nothing is sized to the hardware.
The validator is never invoked — it is an optional local command, not a gate.
The runtime accepts whatever shape it is given and reports healthy. The provider answers, so liveness passes; the shape is simply unverified — and a wrong one (specifically one below the safe band) would manifest as silent truncation or slow requests attributed to the provider, never as a configuration error.
The system therefore had every fact required to check this and never connected them. LLAMA_ARG_ENDPOINT_SLOTS: "true" is already set, so the engine's live slot and per-slot-context shape is queryable. The elected envelope is already a first-class artifact. Nothing compares them.
The consequence is the worst available failure mode: a misconfigured deployment is indistinguishable from a correctly-configured one. On the observed plane the shape happened to be right — and proving that took a multi-session investigation that a boot-time comparison would have answered in one line.
Architectural Reality
llama-server exposes its live configuration; n_slots and n_ctx_slot appear in its own startup log and via the slots endpoint the compose already enables.
The elected envelope is durable — it is the output of #17024 and the input this profile declares it consumes.
Health today answers "can the provider respond?". It does not answer "is the provider shaped the way this deployment intends?" — and only the second question catches a guessed envelope.
A forked compose is not an abuse to be prevented; it is the normal path for any deployment whose topology differs (ingress, extra services, different volumes). The guard has to live at runtime, not in the file.
The Fix (shape)
At boot, read the embedding lane's live shape from the engine and compare it to the elected envelope, with safeProcessingLimitTokens as the floor the comparison must enforce (#17070 files that linkage).
On mismatch, degrade the health surface with a precise, actionable reason — naming observed vs expected per-slot context — rather than reporting healthy and letting the divergence express itself hours later.
Report the observed lane shape in the deployment-state snapshot regardless of match, so the value is visible without a shell on the host.
Consider making the composition validator runnable against a rendered running deployment, not only against a local docker compose config — the deployments most likely to be wrong are the ones nobody renders locally.
Acceptance Criteria
(Substituted 2026-08-14 per @neo-opus-ada's intake, adopted with one added clause; the original ACs are preserved in the correction comment. Authority correction: #17024 — "the elected envelope" — was closed NOT_PLANNED on 2026-08-12, the day before this ticket was written, and no election artifact exists in production code. The comparison authority is the resolved declared configuration plus the safe band (ai/embeddingSafeBand.mjs, shipped by #17070 / PR #17090). Framing correction: #17090's structured HTTP 400 refusal already backstops the corruption path, so this ticket delivers early DRIFT DIAGNOSIS and remote visibility, not a truncation barrier — its close must not claim the guarantee #17090 already owns.)
A one-shot boot-time observer reads the live embedding-lane slot shape (/slots, issued once before the first embedding request — never on the request path, which is exactly the recurring-probe shape PR #17090 removed) and records the comparison.
The comparison runs two independent arms, each naming its numbers in the receipt: (a) declared arm — observed per-slot context and parallelism vs the values the deployment EXPLICITLY declared, carried on a dedicated declaration channel: the raw NEO_PROVIDER_LANE_EMBEDDING_SLOTS and NEO_PROVIDER_LANE_EMBEDDING_CONTEXT_TOKENS_PER_SLOT_REQUIRED inputs are forwarded to the orchestrator and bound to NULL-defaulting leaves, so not-declared is a resolved property (null), never a heuristic — a plane whose leaves resolve null records declared: not-declared and this arm neither runs nor degrades. The consumption-namespace leaves (NEO_LOCAL_MODELS_EMBEDDING_*, non-null defaults — parallel defaults to 1 and matches essentially no real multi-slot lane) are NEVER a comparison authority: a resolved default is indistinguishable from a declaration there, which is exactly the false-degrade the intake reproduced; (b) floor arm — observed per-slot context vs safeProcessingLimitTokens, consumed from ai/embeddingSafeBand.mjs, never re-derived; this arm always runs, declarations or none.
A fired arm degrades health with a reason naming which arm and both numbers; a malformed or unavailable /slots reading records an explicit unobservable posture that neither degrades nor asserts green, and never triggers a restart.
The observed lane shape appears in the deployment-state snapshot on a healthy plane too (projected beside the existing provider blocks, readable through inspect_deployment / get_deployment_state_snapshot — the health-exempt diagnostics #17066 restored).
A correctly-shaped deployment — including one that declares nothing and passes the floor — shows no degradation (negative control).
Regression fixture: a lane rendered below the safe-band per-slot context is flagged at boot, before any embedding request is attempted.
Out of Scope
Changing the election algorithm (#17024) or the shipped per-slot floor.
The healthcheck defect in the same block (#17063) — that one is byte-identical between canonical and fork, and is genuinely ours.
"They should not have forked the compose." Forking is the normal path for a differing topology, and the canonical file is an overlay that does not stand alone. A contract enforced only by substitution syntax is enforced only for people who did not need to change anything.
"The validator already exists." It exists and is optional, local, and pre-deployment. Nothing requires it to have run, and nothing re-checks after a value drifts.
"Health was green, so detection worked." Health answered a narrower question than the one that mattered. The provider genuinely was alive; whether it was shaped right was never asked.
Related
#17024 — elects the fixed-envelope allocation this profile consumes
#17070 — the safe-band floor + truncation detection this check must enforce
Context
ai/deploy/docker-compose.provider-lanes.ymlis explicit about its own contract:It enforces that with fail-closed substitution —
${NEO_PROVIDER_LANE_EMBEDDING_SLOTS:?embedding slots required},…TOTAL_CONTEXT_TOKENS:?,…UBATCH_TOKENS:?,…EMBEDDING_CPUS:?— and ships a validator (ai/scripts/diagnostics/providerLaneComposition.mjs) plus an election runner (ProviderLaneElectionRunner) that benchmarks candidate slot counts against the actual hardware. A per-slot floor appears asNEO_PROVIDER_LANE_EMBEDDING_CONTEXT_TOKENS_PER_SLOT_REQUIRED(8192 in the test fixtures — a fixture value, not a shipped default; see the correction below).That protection is defeated by copying the file.
An external plane was observed on 2026-08-13 running a compose whose
embedding-modelblock is byte-identical to the canonical one — same image digest, same entrypoint, same healthcheck — except that every:?-required value had been replaced with a literal:LLAMA_ARG_CTX_SIZE=131072 # canonical: ${…TOTAL_CONTEXT_TOKENS:?} LLAMA_ARG_N_PARALLEL=4 # canonical: ${…SLOTS:?} LLAMA_ARG_BATCH=131072 # canonical: ${…BATCH_TOKENS:?} LLAMA_ARG_UBATCH=32768 # canonical: ${…UBATCH_TOKENS:?} cpus: "${EMBEDDING_CPU_LIMIT:-6.0}" # canonical: ${…EMBEDDING_CPUS:?}Correction (2026-08-14, replacing this section's original worked example — the original is preserved in the correction comment): the authoritative floor is
safeProcessingLimitTokens(28,672,ai/configBase.mjs) — Neo deliberately sends single embedding inputs up to that size — and against that band the observed plane's 32,768 tokens per slot is correct (the smallest power of two holding a safe-band input; #17070 files the floor linkage). The engine reportedn_slots = 4, n_ctx_slot = 32768. The plane's slow requests (a 9,144-token embed measured ~11 minutes) were later traced to engine thread oversubscription inside the container CPU quota plus a heavyweight healthcheck (#17063) — pathology-contaminated numbers, not evidence of a wrong envelope.The defect this ticket keeps is narrower and fully stands: the values were substituted as literals, so nothing verified them. A deployment sized below the safe band would have run exactly as silently — in the direction that corrupts data instead of merely running slow.
The Problem
The fail-closed
:?guard protects exactly one thing: rendering this file without an elected envelope. It cannot protect against the far more likely operational path — a deployment that vendors its own compose and substitutes literals, whether by copy-paste, by templating, or because the deployment topology needs services the canonical overlay does not include.Once that happens:
The system therefore had every fact required to check this and never connected them.
LLAMA_ARG_ENDPOINT_SLOTS: "true"is already set, so the engine's live slot and per-slot-context shape is queryable. The elected envelope is already a first-class artifact. Nothing compares them.The consequence is the worst available failure mode: a misconfigured deployment is indistinguishable from a correctly-configured one. On the observed plane the shape happened to be right — and proving that took a multi-session investigation that a boot-time comparison would have answered in one line.
Architectural Reality
llama-serverexposes its live configuration;n_slotsandn_ctx_slotappear in its own startup log and via the slots endpoint the compose already enables.The Fix (shape)
safeProcessingLimitTokensas the floor the comparison must enforce (#17070 files that linkage).docker compose config— the deployments most likely to be wrong are the ones nobody renders locally.Acceptance Criteria
(Substituted 2026-08-14 per @neo-opus-ada's intake, adopted with one added clause; the original ACs are preserved in the correction comment. Authority correction: #17024 — "the elected envelope" — was closed NOT_PLANNED on 2026-08-12, the day before this ticket was written, and no election artifact exists in production code. The comparison authority is the resolved declared configuration plus the safe band (
ai/embeddingSafeBand.mjs, shipped by #17070 / PR #17090). Framing correction: #17090's structured HTTP 400 refusal already backstops the corruption path, so this ticket delivers early DRIFT DIAGNOSIS and remote visibility, not a truncation barrier — its close must not claim the guarantee #17090 already owns.)/slots, issued once before the first embedding request — never on the request path, which is exactly the recurring-probe shape PR #17090 removed) and records the comparison.NEO_PROVIDER_LANE_EMBEDDING_SLOTSandNEO_PROVIDER_LANE_EMBEDDING_CONTEXT_TOKENS_PER_SLOT_REQUIREDinputs are forwarded to the orchestrator and bound to NULL-defaulting leaves, sonot-declaredis a resolved property (null), never a heuristic — a plane whose leaves resolve null recordsdeclared: not-declaredand this arm neither runs nor degrades. The consumption-namespace leaves (NEO_LOCAL_MODELS_EMBEDDING_*, non-null defaults —paralleldefaults to 1 and matches essentially no real multi-slot lane) are NEVER a comparison authority: a resolved default is indistinguishable from a declaration there, which is exactly the false-degrade the intake reproduced; (b) floor arm — observed per-slot context vssafeProcessingLimitTokens, consumed fromai/embeddingSafeBand.mjs, never re-derived; this arm always runs, declarations or none./slotsreading records an explicitunobservableposture that neither degrades nor asserts green, and never triggers a restart.inspect_deployment/get_deployment_state_snapshot— the health-exempt diagnostics #17066 restored).Out of Scope
Avoided Traps
Related