LearnNewsExamplesServices
Frontmatter
id17054
titleLM Studio residency mutations overlap across recovery authorities
stateClosed
labels
bugairegressionagent-os
assigneesneo-gpt-emmy
createdAt2:43 PM
updatedAt3:31 PM
githubUrlhttps://github.com/neomjs/neo/issues/17054
authorneo-gpt-emmy
commentsCount1
parentIssue14154
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAt3:31 PM

LM Studio residency mutations overlap across recovery authorities

Closed Backlog/active-chunk-15 bugairegressionagent-os
neo-gpt-emmy
neo-gpt-emmy commented on 2:43 PM

Context

PR #17053 fixes the measured #17051 incident by keeping the Process Supervisor's existing per-task latch held until its LM Studio readiness hook settles. That closes overlapping polls from one supervisor task.

The same destructive helper is also called by the recovery actuator inside the same Orchestrator process, outside that per-task latch:

  • ConfiguredTaskDefinitionsService calls ensureLmsModelsLoaded() from the LMS task's postSpawn readiness hook.
  • RecoveryActuatorService.warmProviderResidency() calls repairProviderRoleSetResidency(), whose LM Studio branch calls ensureLmsModelsLoaded().

Both authorities can therefore enter the same lms unload / lms load sequence concurrently even after #17051 is repaired.

The Problem

The overlap is directly reproducible on current source without LM Studio or network I/O. Two concurrent ensureLmsModelsLoaded() calls were given independent model sets and deferred injected loadModel seams. The measured result was:

{"maxActive":2}

That means the provider helper itself admits overlapping residency mutations. A recovery action can race the supervisor's readiness pass, repeat the exact unload/load cancellation class from #17051, and defeat the caller-local latch.

This is not a request for another residency subsystem. The helper module already serializes its OpenAI-compatible embedding-serving canary with a Promise tail; the missing property is at the destructive LMS residency boundary.

The Architectural Reality

  • ai/services/graph/providerReadinessHelper.mjs owns ensureLmsModelsLoaded(), including every LMS model unload and load.
  • ai/daemons/orchestrator/services/ConfiguredTaskDefinitionsService.mjs is the routine supervisor-readiness caller.
  • ai/daemons/orchestrator/services/RecoveryActuatorService.mjs is the recovery caller.
  • Both services are composed by the same Orchestrator instance and therefore share one JavaScript module instance.
  • The supervisor's _livenessProbeInFlight latch is deliberately task-local. It cannot serialize a separate recovery-actuator call.

Structure-map receipt at filing:

  • ai/services/graph: 36 files; providerReadinessHelper.mjs = 2,006 code LOC.
  • ai/daemons/orchestrator/services: 40 files; ConfiguredTaskDefinitionsService.mjs = 184 LOC, RecoveryActuatorService.mjs = 870 LOC, ProcessSupervisorService.mjs = 695 LOC.

The Fix

Serialize complete ensureLmsModelsLoaded() operations through one process-wide Promise tail in providerReadinessHelper.mjs.

This must be a FIFO queue, not coalescing:

  1. the first caller executes normally;
  2. a concurrent caller waits;
  3. after the predecessor settles, the queued caller performs its own fresh probes with its own models, shape requirements, and authority oracle;
  4. a rejected predecessor cannot poison the queue;
  5. every unload/load remains behind the existing fresh isAuthorityHeld check.

No new service, daemon, config leaf, file lease, or public deployment surface is needed.

Contract Ledger

Target Surface Source of Authority Proposed Behavior Failure / Fallback Evidence
ensureLmsModelsLoaded(options) providerReadinessHelper.mjs concurrent calls execute FIFO, never overlapping their discovery/mutation lifecycle predecessor rejection releases the tail; the next call runs and preserves its own result/error focused concurrent-call unit specs
LMS supervisor readiness ConfiguredTaskDefinitionsService retains its existing payload and result; now composes safely with recovery no new caller contract existing supervisor specs + positive control
LMS recovery repair RecoveryActuatorServicerepairProviderRoleSetResidency() waits behind an active routine repair, then re-probes under its own authority moved authority still refuses at the existing last-boundary check queue + rejection/authority witness

Decision Record impact

Aligned with ADR 0026. Recovery remains authority-gated and bounded; this closes an in-process mutation race between two existing action paths. No ADR-0019 configuration change: callers keep reading the existing reactive SSOT leaves at their current use sites.

Acceptance Criteria

  • Two concurrent ensureLmsModelsLoaded() calls can never execute more than one injected LMS load/unload lifecycle at a time.
  • The queued caller does not begin its provider discovery or mutation work until its predecessor settles.
  • The queued caller retains its own models, context/parallel requirements, collaborators, and return value; calls are serialized, never coalesced.
  • A rejected first call does not poison or skip the second call.
  • A queued recovery call re-evaluates its own isAuthorityHeld oracle before every mutation after it acquires the queue.
  • Sequential existing callers preserve their current results and error behavior.
  • Focused provider-readiness and supervisor suites pass.

Out of Scope

  • Changing LM Studio JIT/TTL settings.
  • Replacing the LMS CLI or moving residency into another service.
  • A cross-process lock for hypothetical external mutators; current production callers are composed in one Orchestrator process.
  • Provider-lane deployment work; its Ollama and dedicated llama.cpp containers do not execute this LMS path.
  • The broader observation-versus-intervention census owned by #16856.

Avoided Traps

  • Do not coalesce callers. A recovery call may carry a different authority oracle or role set; returning another caller's result would forge evidence.
  • Do not add a second supervisor latch. The shared mutation boundary is the only seam that sees every in-process caller.
  • Do not introduce a durable/file lease. The validated callers share one module instance; a filesystem protocol would add failure modes without closing a measured boundary.
  • Do not fold model-shape policy into this repair. This ticket serializes existing decisions; it does not change which model shapes are considered sufficient.

Related

  • Parent series: #14154
  • First measured caller-local repair: #17051 / PR #17053
  • Recovery admission authority: #17012
  • General probe/intervention classification: #16856

Live duplicate sweep: latest 30 open issues plus scoped live searches for LM Studio residency, unload/load overlap, supervisor, and recovery-actuator ownership; no equivalent ticket found. The Agent OS mailbox/A2A query surface was unavailable in this harness at filing time, so no claim was inferred from unread state.

Retrieval Hint: LM Studio readiness recovery actuator ensureLmsModelsLoaded overlapping unload load

tobiu referenced in commit 2d2dc78 - "fix(ai): serialize LM Studio residency mutations (#17054) (#17055)" on 3:31 PM
tobiu closed this issue on 3:31 PM