LearnNewsExamplesServices
Frontmatter
titlefix(ai): verify lms loaded context readiness (#13851)
authorneo-gpt
stateMerged
createdAtJun 23, 2026, 9:39 AM
updatedAtJun 23, 2026, 5:35 PM
closedAtJun 23, 2026, 5:34 PM
mergedAtJun 23, 2026, 5:34 PM
branchesdevcodex/13851-lms-context-readiness
urlhttps://github.com/neomjs/neo/pull/13913
contentTrust
projected
quarantined0
signals[]
Merged
neo-gpt
neo-gpt commented on Jun 23, 2026, 9:39 AM

Resolves #13851

Related: #13874, #13923

Adds the missing post-load LM Studio readiness check: after ensureLmsModelsLoaded() reloads context-configured models, it now verifies observed lms ps --json metadata for loaded context length and configured chat parallel slots before declaring LMS ready. Plain presence-only readiness remains unchanged when no context/parallel gate is configured.

Evidence: L2 (mocked lms ps --json dispatch + focused provider-readiness unit coverage) plus a non-mutating live L3 readiness probe. The live probe confirms the corrected burn-source risk: current LM Studio reports text-embedding-qwen3-embedding-8b loaded at 8192 while Neo's config requires 32768, and the rebased helper flags that model as insufficient. Residual: operator may still run the issue's heavy-session digest / choke-sweep post-merge.

Deltas from ticket

  • Uses the existing orchestrator task path through buildLmsPreloadConfig() and ensureLmsModelsLoaded() instead of adding another orchestrator layer.
  • Verifies the configured chat --parallel slot count when present, preventing LM Studio's default multi-slot KV-cache split from satisfying readiness.
  • Keeps lms ps --json narrowly gated to LMS models with configured contextLengths or parallels, so deployments without context enforcement do not gain a new CLI dependency.
  • Confirms the embedding-on-ingestion caveat from #13923 separately from the REM/chat retry-loop fix in #13922.

Contract Ledger

Target Surface Source of Authority Behavior Evidence
LM Studio model readiness ai/services/graph/providerReadinessHelper.mjs Required LMS models with configured context/parallel gates must report observed loaded metadata that satisfies those gates before readiness is true. npm run test-unit -- test/playwright/unit/ai/scripts/runners/runSandman.spec.mjs --workers=1
Orchestrator LMS task ai/daemons/orchestrator/services/ConfiguredTaskDefinitionsService.mjs Existing post-spawn path passes contextLengths and parallels into ensureLmsModelsLoaded(). Source audit during implementation

Test Evidence

  • node --check ai/services/graph/providerReadinessHelper.mjs
  • node --check test/playwright/unit/ai/scripts/runners/runSandman.spec.mjs
  • npm run test-unit -- test/playwright/unit/ai/scripts/runners/runSandman.spec.mjs --workers=1 - 47 passed
  • node --input-type=module -e '<live lms ps comparator>' - flagged text-embedding-qwen3-embedding-8b observed 8192 required 32768
  • npm run agent-preflight -- ai/services/graph/providerReadinessHelper.mjs test/playwright/unit/ai/scripts/runners/runSandman.spec.mjs
  • node buildScripts/util/check-block-alignment.mjs ai/services/graph/providerReadinessHelper.mjs test/playwright/unit/ai/scripts/runners/runSandman.spec.mjs
  • git diff --check origin/dev..HEAD

Post-Merge Validation

  • On a host with LM Studio running, rerun the issue's operator command sequence: lms unload gemma-4-31b-it && lms load gemma-4-31b-it --context-length 131072 --parallel 1, then verify the choke sweep stops failing at the former 4K boundary.
  • Verify one real heavy session digests to a non-null tri-vector after the readiness gate is active.

Authored by Euclid (GPT-5, Codex Desktop).

neo-gpt
neo-gpt commented on Jun 23, 2026, 2:17 PM

Rebased PR #13913 onto current origin/dev and force-pushed 8f5ab73822.

Fresh evidence after Grace's correction:

  • Focused unit: npm run test-unit -- test/playwright/unit/ai/scripts/runners/runSandman.spec.mjs --workers=1 -> 47 passed.
  • Preflight: npm run agent-preflight -- ai/services/graph/providerReadinessHelper.mjs test/playwright/unit/ai/scripts/runners/runSandman.spec.mjs -> passed.
  • Live non-mutating lms ps --json comparator now flags the deployment-relevant embedding gap: text-embedding-qwen3-embedding-8b observed context 8192, required 32768.

Reviewer slot remains with @neo-opus-grace; no Gemini routing.


tobiu
tobiu APPROVED reviewed on Jun 23, 2026, 5:34 PM

No review body provided.