LearnNewsExamplesServices
Frontmatter
id17051
titleLM Studio readiness hooks overlap and eject configured roles
stateClosed
labels
bugairegressionagent-os
assigneesneo-gpt-emmy
createdAt2:00 PM
updatedAt3:13 PM
githubUrlhttps://github.com/neomjs/neo/issues/17051
authorneo-gpt-emmy
commentsCount1
parentIssue14154
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAt3:13 PM

LM Studio readiness hooks overlap and eject configured roles

Closed Backlog/active-chunk-15 bugairegressionagent-os
neo-gpt-emmy
neo-gpt-emmy commented on 2:00 PM

Summary

The LM Studio supervisor can run multiple model-readiness hooks concurrently. Those hooks each call lms load / lms unload for the configured chat and embedding roles, so they cancel one another and repeatedly eject models that must remain resident.

This is the now-measured implementation leaf of #14154. It is deliberately one small PR, not a new provider-residency subsystem.

Live evidence

On the canonical local Agent OS at fcc0f814 on 2026-08-13:

  • LM Studio logs show lms-cli unloadModel ejecting google/gemma-4-26b-a4b while an embedding request is active, immediately followed by loadModel ... Operation canceled.
  • Memory Core records HTTP 400: {"error":"Model unloaded.."} for the configured Qwen embedding model.
  • A 45-second lms ps --json trace alternated through embedding-only, both roles, empty, chat-only, embedding-only, empty.
  • The host-edge process simultaneously owned LM Studio chat-load, embedding-load, and unload children.
  • Operator measurement: an ordinary embedding averages about 0.87 seconds; a model unload/reload costs more than 10 seconds. The churn therefore dominates every request and can cancel work, rather than being a harmless cache miss.

The host LM Studio setting unloadPreviousJITModelOnLoad=true can amplify the outcome, but it is not the initiating Neo defect: the explicit unload children are descendants of the Neo host-edge supervisor.

Root cause

ProcessSupervisorService.gateRestartOnLivenessProbe() sets _livenessProbeInFlight[taskName], awaits the cheap task.livenessProbe(), then calls runLivenessReadinessHook(...) without returning or awaiting that Promise. Its finally clears the in-flight latch immediately.

LM Studio loads exceed the 15-second supervisor cooldown. A later poll therefore starts a second readiness hook while the first still mutates model residency. ensureLmsModelsLoaded() may unload an exact resident before reloading its configured context/parallel shape, so overlapping hooks destructively race.

This violates the no-churn/hysteresis intent already established by #13948 and the actual-restart boundary in #12262.

Technical boundary

Scoped structure-map results:

  • ProcessSupervisorService.mjs: 695 code LOC
  • ConfiguredTaskDefinitionsService.mjs: 184 code LOC
  • providerReadinessHelper.mjs: 2,006 code LOC

The repair belongs at the supervisor Promise boundary. It does not require another readiness service, scheduler, daemon, or configuration leaf.

Intended change

  1. Make the liveness Promise chain return/await runLivenessReadinessHook(), so the existing per-task in-flight latch covers both liveness and readiness.
  2. Add a deterministic unit witness with a deferred readiness hook: repeated polls, including one after the cooldown, cannot enter a second hook until the first settles.
  3. Preserve existing missing-model and wrong-shape repair, but serialize it. Once both configured roles converge, routine polling must be non-destructive.
  4. Keep the existing provider-lane profile as a negative control: its fixed llama.cpp embedding process and Ollama chat role must remain resident independently; no repair may stop or reload the opposite lane.

Acceptance criteria

  • With liveness resolved and readiness deliberately held beyond one cooldown, repeated supervisor polls invoke readiness exactly once.
  • The in-flight latch clears only after readiness settles or rejects; a subsequent poll can then enter normally.
  • A readiness failure preserves the existing restart/failure policy and does not strand the latch.
  • With both configured LM Studio roles resident at their required shapes, repeated chat and embedding requests plus routine supervisor polls produce zero lms unload calls.
  • No overlapping readiness hook unloads or reloads a configured role while the first readiness hook remains unsettled.
  • The split provider profile retains both lanes across repeated requests; restarting/repairing one lane does not eject the other.
  • Focused ProcessSupervisor and provider-readiness unit suites pass.

Non-goals

  • Reworking LM Studio's own JIT/TTL settings.
  • Changing configured context sizes or model choices.
  • Retrying failed embeddings as a substitute for stopping the ejection.
  • Adding a new residency framework, deployment diagnostic, or benchmark.
  • Solving unrelated provider contention or the external CPU-runner incident.

Relationships

  • Concrete implementation leaf of #14154.
  • Restores the contract intended by #13948.
  • Related historical supervisor boundary: #12262.
tobiu referenced in commit 920940e - "fix(ai): serialize LM Studio readiness hooks (#17051) (#17053)" on 3:13 PM
tobiu closed this issue on 3:13 PM