LearnNewsExamplesServices
Frontmatter
id13692
titleChroma dynamic embedding-function fans batch adds onto the interactive 15s path (should route through embedTexts)
stateClosed
labels
bugaiarchitecture
assigneesneo-opus-grace
createdAtJun 21, 2026, 3:40 AM
updatedAtJun 21, 2026, 4:58 AM
githubUrlhttps://github.com/neomjs/neo/issues/13692
authorneo-opus-grace
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJun 21, 2026, 4:58 AM

Chroma dynamic embedding-function fans batch adds onto the interactive 15s path (should route through embedTexts)

Closed v13.1.0/archive-v13-1-0-chunk-4 bugaiarchitecture
neo-opus-grace
neo-opus-grace commented on Jun 21, 2026, 3:40 AM

Context

Surfaced during a live heavy-maintenance incident (Refs #13624): the add_memory WAL-drain embedder logged Batch embed exhausted 6 attempts (openAiCompatible request timed out after 15000ms) repeatedly while the github-workflow-sync held the exclusive-heavy lease, starving the single local embedder. V-B-A traced the 15000ms to its source.

Root cause (V-B-A'd to source)

ai/services/shared/vector/chromaClientPrimitives.mjs:53 — the Chroma dynamic embedding-function behind every collection.add():

generate: async (texts) =>
    Promise.all(texts.map(text => TextEmbeddingService.embedText(text, provider)))

fans every batch out as parallel embedText calls — the interactive path, which uses contentionTimeoutMs (15000, config:186) as requestTimeoutMs (TextEmbeddingService.mjs:394, error thrown at :266). It bypasses embedTexts#embedOpenAiCompatibleBatch, the batch path built for exactly this (1-hour timeout via DEFAULT_OPENAI_COMPATIBLE_REQUEST_TIMEOUT_MS, sequential chunks of batchEmbeddingChunkSize, with a yield).

Two compounding faults in that one line:

  1. Wrong timeout — bulk Chroma adds get the 15s interactive cap, not the 1-hour batch cap. embedTexts' own JSDoc (TextEmbeddingService.mjs:423) states batches must "not be cut short by the interactive contention timeout" — but the Chroma path circumvents embedTexts entirely.
  2. Self-inflicted contention stormPromise.all hits the single local embedder with the whole batch at once → the requests contend with each other → the 15s contention timeout trips. The batch path walks chunks sequentially → no storm.

Confirmed chain: WAL drain (drainCycle.mjs:89 collection.add) → Chroma generateembedText (15s). Almost certainly also hits the github-sync Stage-2 graph ingestion (same Chroma path).

Fix

Route the batch through the existing batch path:

generate: async (texts) => TextEmbeddingService.embedTexts(texts, provider)

embedTexts returns ordered number[][] — exactly Chroma's generate contract (verified: #embedOpenAiCompatibleBatch returns data.sort(byIndex).map(d => d.embedding); embedText returns result.data[0].embedding, so both yield number[][]). The WAL drain's per-record isolation pass (drainCycle.mjs:107) still surfaces poison records.

Acceptance Criteria

  • createDynamicTextEmbeddingFunction.generate routes batches through embedTexts (1-hour batch path, sequential chunks), not Promise.all(map(embedText)).
  • Regression test: a multi-text generate() exercises the batch path / does NOT apply the interactive contentionTimeoutMs (assert via the timeout or an injected seam).
  • The interactive embedText 15s path is unchanged (single latency-sensitive embeds keep it).

Out of scope

Raising contentionTimeoutMs — the path-swap removes the contention that made 15s fail; revisit only if interactive single-embeds still contend after this lands.

Refs #13624 (heavy-maintenance incident), #13498 (who_is_online canary on the same interactive path — leaf-4, may co-land separately).

tobiu referenced in commit 6a51297 - "fix(ai): route Chroma batch embeds through embedTexts, not the interactive path (#13692) (#13695) on Jun 21, 2026, 4:58 AM
tobiu closed this issue on Jun 21, 2026, 4:58 AM