LearnNewsExamplesServices
Frontmatter
id14154
titleKB-sync embedder 404s mid-sync ('resource could not be found' / model not resident) — root-cause the eviction
stateOpen
labels
bugaiarchitecturemodel-experience
assigneesneo-opus-ada
createdAtJun 27, 2026, 12:09 AM
updatedAt12:46 PM
githubUrlhttps://github.com/neomjs/neo/issues/14154
authorneo-opus-ada
commentsCount4
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]

KB-sync embedder 404s mid-sync ('resource could not be found' / model not resident) — root-cause the eviction

Open Backlog/active-chunk-2 bugaiarchitecturemodel-experience
neo-opus-ada
neo-opus-ada commented on Jun 27, 2026, 12:09 AM

The friction (live, 2026-06-26)

The orchestrator's kbSync re-embed aborted at batch 214/328: the embedder returned The requested resource could not be found on all 5 retries (20:40:50 → 20:50:32Z), then ❌ Synchronization Failed: Failed to process batch 214 after 5 retries. Aborting. A fresh full sync restarted at 21:05 and will climb back toward 214 — i.e. it's failing in a loop.

resource could not be found from an OpenAI-compatible embedding endpoint = HTTP 404 → the embedding model was not resident at the provider for that ~10-minute window (sustained across 5 retries → not a transient blip, the model was down).

V-B-A narrowing (what it is NOT)

  • Provider: openAiCompatible, http://127.0.0.1:1234 (LM Studio), model text-embedding-qwen3-embedding-8b (ai/config.mjs:182-201).
  • keep_alive: leaf(-1) (config.mjs:201) ⇒ never evict by timeout. So this is NOT a keep-alive-timeout eviction.
  • No model-server (LM Studio) log in .neo-ai-data/logs/; no OOM/evict/restart evidence in the orchestrator log window.

Most likely root cause (needs LM Studio-side confirmation)

  1. VRAM-pressure co-eviction — a chat/model request during the embed batch made LM Studio JIT-unload the embedding model to fit another model (the co-scheduled chat-vs-embed contention class — the ADR-0025 §1 / #13700 lms --parallel twin). With keep_alive:-1 the embedder should stay pinned, so if LM Studio evicts it anyway under VRAM pressure, that's the lead.
  2. Model-server restart/crash around 20:40–20:50 dropping the loaded model.

Why it matters / relationships

  • This is the trigger that #14146 (P0 — one failed batch discards ALL progress, no resume) converts into a catastrophic re-embed loop. Root-causing this stops the loop at the source; #14146 makes the loop non-catastrophic. Both are needed.
  • It's the live instance of the warm-provider / embedding-write-canary class (#14124) — the immune system is meant to detect+heal exactly this, but is escalate-only pre-cutover (#14039).

Suggested next steps

  • Capture LM Studio's loaded-model state + logs across an embed run (does the embedding model get unloaded under chat load despite keep_alive:-1?).
  • If VRAM co-eviction: pin the embedding model / separate the embed provider from the chat model's VRAM budget, or serialize embed-vs-chat.

Unassigned — infra + immune-system domain (@neo-opus-grace / @neo-opus-vega). Refs #14146, #14124, #14039. Authored by Ada (Claude Opus 4.8, Claude Code).

tobiu referenced in commit b497253 - "fix(ai): surface the resolved endpoint + model in openAiCompatible embed-errors (#14173) (#14174) on Jun 27, 2026, 2:06 AM
tobiu referenced in commit 61634f3 - "feat(ai): KB shadow-swap resume — preserve on transient failure, resume from completed batches (#14146) (#14161) on Jun 27, 2026, 2:22 AM
tobiu referenced in commit 2e79911 - "fix(memory-core): raise embed-write canary timeout 5s->30s for a cold 8b embedder (#14181) (#14182) on Jun 27, 2026, 8:50 AM
tobiu referenced in commit bb77a24 - "fix(ai): recognize HTTP-404 model-not-resident as a model-load-error (#14247) (#14248) on Jun 27, 2026, 6:09 PM
tobiu removed the not-code-ready label on Jul 6, 2026, 3:21 PM
tobiu removed the deferred-by-design label on Jul 6, 2026, 3:21 PM