LearnNewsExamplesServices
Frontmatter
id16870
titleA short native-Ollama embedding response binds vectors to the wrong chunk ids
stateClosed
labels
bugaiarchitecture
assigneesneo-gpt-emmy
createdAtAug 10, 2026, 11:47 AM
updatedAtAug 10, 2026, 1:14 PM
githubUrlhttps://github.com/neomjs/neo/issues/16870
authorneo-opus-ada
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 10, 2026, 1:14 PM

A short native-Ollama embedding response binds vectors to the wrong chunk ids

Closed Backlog/active-chunk-14 bugaiarchitecture
neo-opus-ada
neo-opus-ada commented on Aug 10, 2026, 11:47 AM

⚠️ Corrected 2026-08-10 — the causal claim in this ticket is RETRACTED

This ticket said a short response produces silently wrong rows in the corpus. It does not. @neo-gpt probed the installed ChromaDB 3.5.0 client and a mismatched record set throws ChromaValueError: Unequal lengths for fields … with apiCalled=false; I verified it at chromadb.legacy-esm.js:1080 and found the empty case is refused too (Non-empty lists are required for …). The store validates before any API call.

The original text is retained below unedited. It was wrong, it circulated, and a reader who saw only corrected prose would not know to distrust the neighbouring reasoning.

The defect is real but its class changed: a contract defect, not a data-corruption one. The guard's value is diagnostic locality — the failure is named at the input count by the layer that knows both numbers, rather than surfacing three layers down as an opaque store error — plus protection for callers that do not terminate at Chroma.

How the wrong conclusion survived: the spec's collection double accepted any {ids, embeddings} pair, which deleted Chroma's mandatory refusal rather than simulating it. Every assertion downstream of that removal was a property of the double. The double now models both refusals and carries its own control so a permissive version cannot return.

Generalisable: when a test replaces a boundary, ask what that boundary refuses, not only what it records. A permissive double does not fail to simulate the real thing — it removes an invariant.

Context

A deployment runs NEO_EMBEDDING_PROVIDER=ollama with qwen3-embedding and has had an empty Knowledge Base for two months. @neo-opus-grace established that our proof did not cover that provider: batch-failure isolation, lease-yield persistence and rebuild recovery were all measured on openAiCompatible, and no full ingest sweep had ever been run on ollama-native.

Asking the CI half of that question found a live defect on the untested axis.

Live latest-open sweep at 2026-08-10T08:33:41Z; A2A claim sweep over the herd window. #16853 (early abort strands a runner) and #16849 (Ollama.stream() has no timeout) govern how a request ends; this is what its response is allowed to mean.

The Problem

TextEmbeddingService.embedTexts(texts, 'ollama') returned result.embeddings || [] with no length check.

Native ollama's /api/embed returns a parallel array with no per-item index. Length is therefore the only thing binding a vector to its input. A response carrying two vectors for three inputs shifts every later vector onto its neighbour's id:

id vector it receives
chunk-0 the vector computed for chunk-1
chunk-1 the vector computed for chunk-2
chunk-2 nothing

No length mismatch surfaces and no error is raised. VectorService.embedChunks feeds the array straight into collection.upsert({ids, embeddings, metadatas}), so it reaches the corpus.

A wrong row is worse than a failed batch. The batch retries; the row is believed. A wrong vector under a correct id is a permanently wrong search result that nothing will ever flag.

The || [] was the same defect one degree further: a malformed response became zero vectors and no error, so a sweep reports progress while the corpus stays empty — the two-month signature exactly.

Reproduced, not reasoned: three ids upserted against two vectors, asserted at the collection.

The Architectural Reality — why the suite could not see it

Every KB ingest spec replaces TextEmbeddingService.embedTexts with a fake. That is correct for what those specs test: VectorService's batch and slice bookkeeping is genuinely above the provider seam, which is why the provider-independence argument is sound. But it means the suite never dispatches through the seam, and no spec under test/…/knowledge-base/ drove embeddingProvider: 'ollama' at all.

Two halves were each covered and their composition was not:

  • above the seam — VectorService bookkeeping, stubbed → green on both providers
  • below the seamembedTexts(texts,'ollama') shape → covered in TextEmbeddingService.spec.mjs
  • the joinembedChunks with the provider set to ollama → nothing

That is precisely why "provider-independent by construction" could not be told apart from "nobody asked". The bookkeeping is provider-independent. The seam was not.

openAiCompatible received this exact guard already (toOrderedEmbeddings, refusing sparse and non-dense responses). This is its twin, on the path a deployment is more likely to run.

The Fix

Validate in the ollama batch branch of embedTexts: the response must be an array whose length equals texts.length, or throw naming both counts. texts.length is derived from what was sent, never from what came back, so a short response cannot define its own correctness.

!== rather than < deliberately: it also catches a longer response, which would shift vectors the other way.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
embedTexts(texts, 'ollama') return TextEmbeddingService number[][] of exactly texts.length throws naming returned-vs-expected counts; no empty-array fallback inline rationale short-response and missing-field specs driving the real seam
every embedTexts caller 5 call sites (below) unchanged on the happy path; a malformed response now throws instead of resolving empty n/a caller census in Deltas

Acceptance Criteria

  • A short ollama response is refused rather than bound to ids by position. — three inputs, two vectors: nothing reaches the collection and the sweep aborts.
  • A missing embeddings field is refused rather than becoming an empty array. — the malformed-response case aborts instead of resolving.
  • Rows land end to end through the ollama seam — the happy path asserts spy.upserts[0].ids equals the chunk ids with one vector per id in order, not that a call was made. A dispatch-counting spec passes against a path that upserts nothing, which is the incident's own signature.
  • The specs drive the REAL seam: ollamaProvider is stubbed below the seam and embedChunks runs with embeddingProvider: 'ollama'. Stubbing embedTexts would certify the stub.
  • Shared-singleton mutation is restored in cleanup and the spec is serial — the order-dependent pollution class this suite has hit before.
  • A LONGER response is refused too, so !== cannot silently regress to <. (Added 2026-08-10 — the criterion followed the test, not the reverse.) — three vectors for two inputs, asserting the exact refusal message. Authored by @neo-gpt-emmy (bf0d508a80), and it closes a real hole: @neo-opus-grace raised the longer-response case in pre-review, I recorded it in the PR body as a design note and never wrote the test, so the guard's upper bound was unpinned and !==< would have passed every spec on the branch.
  • Test setup isolates through the shipped snapshotAiConfig primitive, not a hand-rolled save/restore — for both the MC provider selector and the KB batch leaves. Authored by @neo-gpt-emmy (75c10d93d0) after I converted only the first and left the second on Object.assign. The primitive captures by resolved value (the Provider's getOwnPropertyDescriptor trap misses leaves its get trap resolves) and throws on a non-resolving leaf; a hand-rolled capture tolerates that silently and then cannot undo it.
  • Coverage fails against today's code and passes against the repair. — the short-response spec reproduces the wrong-binding upsert on the pre-fix tree.

Out of Scope

  • A real-model plane run. These specs prove the code path binds vectors to ids correctly; they cannot prove qwen3-embedding answers correctly on CPU-only hardware. That is @neo-gpt-emmy's plane sequencing and is deliberately not claimed here — treating CI green as plane-proven would repeat the overclaim this ticket exists because of.
  • How a request ends#16853, #16849, #16860.
  • Restoring the reason through the abort. On dev the total-outage arm mints a bare Error, so an operator receives "batch failed" and the refusal's message is lost. PR #16854's cause-preservation fixes that; the data-corruption fix here stands alone and must not wait for it.

Avoided Traps

  • Stubbing embedTexts in the new specs. Would have certified the stub — the exact defect diagnosed in the existing suite.
  • Asserting a call was made. Passes against a path that upserts nothing.
  • < instead of !==. Silently accepts a longer response.
  • Keeping an empty-array fallback "for safety". It is what let a malformed response report success.

Related

#16706, #16843, #16853, #16854, #16826 (the openAiCompatible twin).

Origin Session ID: 87f453f9-aa80-4487-9ed1-b5d91e052c43

Retrieval Hint: ollama embedding response length guard parallel array positional binding KB ingest seam

Authored by Ada (Claude Opus 5, Claude Code).

tobiu referenced in commit 5d10491 - "fix(memory-core): refuse a short native-Ollama embedding response (#16870) (#16871) on Aug 10, 2026, 1:14 PM
tobiu closed this issue on Aug 10, 2026, 1:14 PM