LearnNewsExamplesServices
Frontmatter
id12117
titleloadLmsModel: thread --context-length from localModels.chat.contextLimitTokens (close L4 silent-context-mismatch gap exposed by post-merge Sandman validation)
stateClosed
labels
enhancementaiarchitecturemodel-experience
assigneesneo-opus-ada
createdAtMay 28, 2026, 12:51 AM
updatedAtMay 28, 2026, 1:59 AM
githubUrlhttps://github.com/neomjs/neo/issues/12117
authorneo-opus-ada
commentsCount0
parentIssue12101
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtMay 28, 2026, 1:59 AM

loadLmsModel: thread --context-length from localModels.chat.contextLimitTokens (close L4 silent-context-mismatch gap exposed by post-merge Sandman validation)

Closed v13.0.0/archive-v13-0-0-chunk-14 enhancementaiarchitecturemodel-experience
neo-opus-ada
neo-opus-ada commented on May 28, 2026, 12:51 AM

Premise

PR #12112 landed proactive lms load <model> invocation on orchestrator-managed lms task spawn via ai/services/graph/ProviderReadinessHelper.mjs:loadLmsModel. The helper invokes execFile('lms', ['load', model], ...) WITHOUT passing --context-length. This is the L4 validation gap surfaced in PR #12112's ## Post-Merge Validation and during the actual post-merge Sandman run (2026-05-27):

Empirical anchor (post-PR-#12115-merge orchestrator restart + ai:run-sandman):

  • LM Studio with gemma-4-31b-it loaded via lms load (default modelfile context ~4K-8K)
  • Sandman REM cycle: all 10 sessions returned silent empty-response → PR #12113 detection correctly fired context-overflow friction for all 10 sessions
  • Direct V-B-A: curl -sN /v1/chat/completions with tiny prompts to the same LM Studio endpoint returns content cleanly → confirms LM Studio + model alive
  • Root cause: loaded-context-cap < composed-prompt-size; even 18K-char (~4.5K-token) sessions overflow the modelfile-default context window

Operator manually ran lms load gemma-4-31b-it --context-length 262144 to recover — that command is exactly what loadLmsModel SHOULD have done automatically per the substrate intent of #12090.

Substrate shape

The neo-side aiConfig.localModels.chat.contextLimitTokens (PR #12115's role-keyed split — default 262144) is the operator's declared chat-context-window threshold. The lms-side loaded-model context-cap MUST be ≥ this value or every chat invocation silently fails the loaded-cap pre-check. These two layers are currently INDEPENDENT.

loadLmsModel is the substrate boundary where they SHOULD be tied together: the orchestrator already knows the neo-side threshold (localModels.chat.contextLimitTokens flows through Orchestrator.lmsModels getter + TaskDefinitions postSpawn hook), so it can pass it directly to lms load --context-length <N>.

Prescription

  1. ProviderReadinessHelper.loadLmsModel: accept new contextLength param; when present, append '--context-length', String(contextLength) to the execFile args.
  2. ProviderReadinessHelper.ensureLmsModelsLoaded: accept contextLength param; thread it through to each loadLmsModel(model, {contextLength}) call.
  3. TaskDefinitions.mjs lms task postSpawn hook: pass contextLength: aiConfig.localModels.chat.contextLimitTokens (chat-role since LM Studio's loaded gemma is the chat consumer; embedding context-cap for the embedding model would be a parallel threading).
  4. Sibling defense — embedding context: pass contextLength: aiConfig.localModels.embedding.contextLimitTokens for the embedding-model load invocation in the same ensureLmsModelsLoaded orchestration (per PR #12112's design which loads BOTH chat + embedding models).

Contract Ledger

Surface Before After Consumer
loadLmsModel(model, options) execFile('lms', ['load', model], ...) — modelfile-default context (~4K-8K for gemma-4-31b-it) execFile('lms', ['load', model, '--context-length', String(contextLength)], ...) when contextLength provided Orchestrator lms task postSpawn hook
ensureLmsModelsLoaded(options) No context-length awareness New contextLength param (or per-model object keyed by role) TaskDefinitions chat + embedding model load invocations
Loaded-cap / neo-threshold alignment Independent — operator must manually align Orchestrator-owned via threading from aiConfig.localModels.{chat,embedding}.contextLimitTokens Eliminates the L4 silent-context-mismatch failure mode

Acceptance Criteria

  • AC1: loadLmsModel accepts contextLength option; when provided, appends '--context-length', String(contextLength) to execFile args; when omitted (legacy callers), behavior unchanged
  • AC2: ensureLmsModelsLoaded accepts per-model context-length config (or a single value if treating chat+embedding uniformly); passes to each loadLmsModel invocation
  • AC3: TaskDefinitions.mjs lms task postSpawn hook reads aiConfig.localModels.chat.contextLimitTokens for the chat model load and aiConfig.localModels.embedding.contextLimitTokens for the embedding model load
  • AC4: Unit tests: loadLmsModel with contextLength asserts execFile args contain '--context-length' + numeric value; ensureLmsModelsLoaded threading covered via stubbed loadModel injection
  • AC5: Orchestrator restart with lmsEnabled=true + neo's chat context-limit 262144 loads gemma chat model with --context-length 262144 → Sandman REM cycle no longer silent-empty on context-mismatch grounds

Related

  • Parent: Epic #12101 (greenfield aiConfig substrate redesign — localModels role-keyed split lives here)
  • Source: PR #12112 (#12090 lms proactive load — established the postSpawn hook + ensureLmsModelsLoaded substrate; this ticket completes the threading)
  • Sibling: #12114 (role-keyed context-limits — established the localModels.{chat,embedding}.contextLimitTokens source-of-truth this ticket threads from)
  • Sibling: #12116 (ConsumerFrictionHelper loud-fail — same defense-in-depth class for the consumer-side telemetry)
  • Empirical anchor: post-merge 2026-05-27 Sandman validation surfaced 10 silent-empty sessions before operator manually lms load --context-length 262144 recovered

Discipline anchor

PR #12112 review cycle-1 flagged the loadLmsModel no-timeout concern as a depth-floor challenge ("Not a merge-blocker — the failure mode is 'boot stalls visibly' rather than 'boots into wrong state'"). The no-context-length is the same class of L4 gap, just with a DIFFERENT failure mode — "boots into context-mismatched state with silent downstream failure" rather than "boots into wrong state". Both are post-merge-validation-grade gaps that the operator surfaced via dogfooding. Closing this ticket completes the L4 lms-proactive-load substrate intent of #12090.

tobiu referenced in commit 6906a97 - "feat(ai): thread --context-length through loadLmsModel from localModels role-keyed limits (#12117) (#12119) on May 28, 2026, 1:59 AM
tobiu closed this issue on May 28, 2026, 1:59 AM