LearnNewsExamplesServices
Frontmatter
id13851
titlefix(ai): orchestrator must verify a loaded model''s context window + free context, not just residency (gemma-4-31b resident at 4096 ≠ configured 131072 → all tri-vector extraction chokes → golden path starves)
stateClosed
labels
bugaiarchitecture
assigneesneo-gpt
createdAtJun 22, 2026, 4:58 AM
updatedAtJun 23, 2026, 5:34 PM
githubUrlhttps://github.com/neomjs/neo/issues/13851
authorneo-opus-grace
commentsCount2
parentIssue13624
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJun 23, 2026, 5:34 PM

fix(ai): orchestrator must verify a loaded model's context window + free context, not just residency (gemma-4-31b resident at 4096 ≠ configured 131072 → all tri-vector extraction chokes → golden path starves)

Closed v13.1.0/archive-v13-1-0-chunk-5 bugaiarchitecture
neo-opus-grace
neo-opus-grace commented on Jun 22, 2026, 4:58 AM

Root Cause (V-B-A, 2026-06-22)

The Computed Golden Path has been empty/stale because nothing digests — and the reason is a model-load context mismatch, not any symptom the arc has chased (defer #13843, chunk #12073, frontier #13801, candidate-pool #13837).

lms ps (live): gemma-4-31b-it is resident, IDLE, 19.70 GB, CONTEXT 4096, PARALLEL 4. Neo config (ai/config.mjs:235-236): contextLimitTokens: 131072 (half gemma's native 256K), safeProcessingLimitTokens: 100000. The model is loaded at 4,096 tokens — 32× below the configured 131,072. Host is 128 GB / 82% free / 0 pageouts → not VRAM, not capacity, not contention (it is idle).

Consequence: every tri-vector extraction prompt (system + schema + any input) overflows 4,096 → empty / "Operation canceled"SemanticGraphExtractor returns null → session never graphDigested → no fresh graph nodes → Golden Path has no candidates. A choke-sweep confirms even 5,000 chars chokes.

The 4,096 is LM Studio's JIT-load default: a direct openAiCompatible request loaded the model at 4,096 instead of the orchestrator preloading it at 131,072 (TTL 59m/1h fits a recent JIT-load).

The Gap

The orchestrator intends to load correctly — Orchestrator.mjs:285 buildLmsPreloadConfig(AiConfig)lmsContextLengthslms load (ai/config.mjs:627). But the post-spawn readiness hook (ProcessSupervisorService:257-321) verifies residency ("the model is resident"), NOT context adequacy ("resident at the configured context window, with enough free"). So a JIT-load at 4,096 — or a preload that did not apply the context length — passes readiness as "ready" while being unusable for extraction.

The Fix

The orchestrator MUST ensure a loaded model has the right amount of context free, not merely that it is resident:

  1. Readiness verifies the resident model's actual loaded context window ≥ the configured contextLimitTokens (via lms ps / loaded-model metadata).
  2. Account for PARALLEL slot-splitting — the per-request usable context must meet the requirement, not just the nominal total.
  3. If a model is resident at an insufficient/wrong context (e.g. 4,096 vs 131,072), reload it at the configured context before marking ready.
  4. Prevent JIT-loads at the LM Studio default from satisfying readiness — the orchestrator owns the load and its context.

Contract Ledger

Target Surface Source of Authority Behavior Fallback Evidence
Orchestrator model-readiness ProcessSupervisorService / Orchestrator preload verify loaded context ≥ contextLimitTokens (+ free-context per PARALLEL); reload if insufficient reload at configured context; hard readiness failure if unachievable the choke-sweep stops choking at large sizes after enforcement

Acceptance Criteria

  • Confirm the root (fix-test, operator-runnable): lms unload gemma-4-31b-it && lms load gemma-4-31b-it --context-length 131072 → re-run /tmp/tri-vector-choke-sweep.mjs → larger sizes stop choking.
  • Readiness verifies the resident model's loaded context window ≥ configured contextLimitTokens — not just residency.
  • Free-context-per-PARALLEL accounted for (per-request usable context meets the requirement).
  • A wrong/insufficient-context resident model is reloaded at the configured context before "ready".
  • JIT-loads at the LM Studio default context cannot satisfy readiness.
  • Live evidence: a real heavy session (e.g. issue-13012.md) digests (non-null tri-vector) after the fix.

Out of Scope / Supersedes

  • Supersedes the premise of #13843 (tri-state defer): sessions are NOT un-digestible — the model was loaded with a 4,096 context. At the correct 131,072 they digest; deferring discards digestible work.
  • Reframes #12073 / #12439 (chunk-and-reduce): chunking is only needed for sessions genuinely exceeding the correct 131,072 context — not the 4,096 misconfig. The "all shapes choke" experiment measured the 4,096-loaded model, not gemma's real ceiling.
  • The idle-window / serialization (contention) load-reduction stays #13834 / #13624 / #13846 — complementary but distinct (this fixes the context-window even at zero contention).

Related

  • The actual root of #13750 (golden-path PRIO-zero, empty/stale handoff).
  • Architecture: #13846 (target-arch ADR).
  • Supersedes premise: #13843. Reframes: #12073, #12439.

Origin Session ID: e96a0867-4c28-4877-b2f4-f56173ee9fd1

tobiu referenced in commit 4b1c150 - "fix(ai): verify lms loaded context readiness (#13851) (#13913)" on Jun 23, 2026, 5:34 PM
tobiu closed this issue on Jun 23, 2026, 5:34 PM