LearnNewsExamplesServices
Frontmatter
id14247
titleTextEmbeddingService: recognize HTTP-404 (model-not-resident) as a model-load-error — retry-with-wait, not blind-abort
stateClosed
labels
bugaiarchitecture
assigneesneo-opus-ada
createdAtJun 27, 2026, 5:13 PM
updatedAtJun 27, 2026, 6:09 PM
githubUrlhttps://github.com/neomjs/neo/issues/14247
authorneo-opus-ada
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJun 27, 2026, 6:09 PM

TextEmbeddingService: recognize HTTP-404 (model-not-resident) as a model-load-error — retry-with-wait, not blind-abort

Closed v13.1.0/archive-v13-1-0-chunk-7 bugaiarchitecture
neo-opus-ada
neo-opus-ada commented on Jun 27, 2026, 5:13 PM

The gap

TextEmbeddingService.#postOpenAiCompatible's retry guard (isModelLoadError, ai/services/memory-core/TextEmbeddingService.mjs:511) matches only HTTP-400 model-load shapes:

  • Shape A: HTTP 400 + Model was unloaded (JIT-unload-then-queued request)
  • Shape B: HTTP 400 + Failed to load model + Operation canceled (JIT-warm-load canceled)

An HTTP-404 (...HTTP 404: The requested resource could not be found... [endpoint=…, model='…'] — the model not resident at the provider) does not match, so it skips the unloadRetriesLeft model-load-wait path and exhausts the remaining blind POSTs, then aborts the batch.

Why it matters

This is the live #14154 trigger: a ~10-minute window where LM Studio had evicted the embedding model returned HTTP-404 across all retries, so the kbSync batch aborted (and, pre-#14146, discarded all progress). The model-load-wait retry — which waits unloadRetryDelayMs between attempts, giving the provider time to reload — is exactly the right response to a model-not-resident 404, but the guard doesn't recognize it. This is the Neo-side self-heal-not-abort mitigation (the env-root-cause — VRAM co-eviction — is #14154's separate infra investigation). Identified as sub-fix #3 in #14146 (closed; the checkpoint/resume half landed, this retry-guard half did not).

The fix

Add Shape C to isModelLoadError: err.message.includes('HTTP 404') (a 404 on /v1/embeddings = model not resident; the error body already carries the endpoint + model for diagnosability). A 404 then triggers the existing bounded model-load-wait retry, and stays fail-loud on a genuinely-permanent 404 (the bounded unloadRetriesLeft exhaust and the batch fails with the diagnostic message — no infinite loop). Update the Shape A/B/C log discriminator.

Acceptance

  • An HTTP-404 embedding error triggers the model-load-wait retry (not a blind abort), bounded by unloadRetriesLeft.
  • A permanent 404 still fails loud after the bounded retries (no infinite loop).
  • Unit test: a 404-shaped error is classified as a model-load-error and retried with wait; existing Shape A/B behavior unchanged.

Scope

TextEmbeddingService.#postOpenAiCompatible retry-guard only — internal retry logic, no consumed public surface (no Contract Ledger). Refs #14154 (incident + env-root-cause), #14146 (sub-fix #3 origin), #14124 (warm-provider / embed-write-canary class).

Authored by Ada (@neo-opus-ada · Claude Opus 4.8, Claude Code).

tobiu referenced in commit bb77a24 - "fix(ai): recognize HTTP-404 model-not-resident as a model-load-error (#14247) (#14248) on Jun 27, 2026, 6:09 PM
tobiu closed this issue on Jun 27, 2026, 6:09 PM