Context
#16780 requires persistence failures to stop causing unbounded repeated embedding work. #16875 and merged PR #16876 made that non-convergence observable, but they explicitly left retaining vectors across a failed write out of scope.
Current dev still places TextEmbeddingService.embedTexts() and collection.upsert() in the same retry try inside VectorService.embedChunks(). A transient vector-write failure therefore discards an already-successful embedding result and re-submits the identical texts to the provider on the next retry.
A measured two-text / three-write-attempt falsifier produces 6 provider submissions on current dev; retaining the successful result across write-only retries produces 2. The implementation prototype is on ada/16780-embed-paid-once at 9ea462ac02cc91f9e963ba79fdeb97208a0dd69b and must be rebased onto current dev before PR creation.
Memory mining surfaced the prior #16875 decision record (session 87f453f9-aa80-4487-9ed1-b5d91e052c43): detection was delivered, while survival across a failed write was deliberately deferred. A final duplicate sweep at 2026-08-11T11:44:26.702Z covered the 20 newest open GitHub issues, the 30 newest all-state A2A messages, and the local issue/discussion corpus; it found no separate live owner for this repair.
This ticket does not claim to explain or cure the permanent 400% Ollama CPU incident. It removes a separately verified redundant-work multiplier.
The Problem
The retry boundary currently conflates two different failures:
- the embedding provider failed, so provider work must be retried;
- embedding succeeded but persistence failed, so only the write must be retried.
Treating both as one attempt re-buys deterministic vectors, inflates provider activity, and can multiply expensive batch work by maxRetries while the collection makes no progress.
Architectural Reality
textsToEmbed and ids are already stable for the batch.
- A successful embedding array is valid input for each retry of the same
collection.upsert().
- JavaScript assignment does not occur when the awaited provider promise rejects, so a nullable result cached with
??= preserves ordinary provider-failure retries.
result.embedded must still advance only after persistence succeeds.
- The existing
VectorService.leaseYield.spec.mjs negative control proves an ordinary embedding failure remains retryable.
- This is a local retry-boundary correction: no AiConfig, public API, worker topology, schema, or service-ownership change.
The Fix
- Hoist a nullable embedding result outside the persistence retry loop.
- Resolve it with
embeddings ??= await TextEmbeddingService.embedTexts(...).
- Reuse that exact result for write-only retries.
- Preserve provider retry behavior when embedding itself rejects.
- Add a mutation-sensitive production-path test at the provider-submission boundary.
Contract Ledger
| Surface |
Required behavior |
Edge / fallback |
Evidence |
VectorService.embedChunks() provider step |
A successfully computed batch is purchased once across retries of its matching write. |
A provider rejection leaves no cached result and remains retryable. |
New transient-upsert falsifier + existing ordinary-provider-failure retry control. |
collection.upsert() |
Continues to retry according to maxRetries, using the same ids, metadata, and vectors. |
Exhausted write retries retain existing failure accounting. |
Three-attempt write fixture. |
result.embedded |
Advances only after the matching upsert succeeds. |
Failed persistence never reports embedded progress. |
Receipt assertion after the third successful write. |
Decision Record Impact
None. This narrows an existing retry boundary without changing configuration authority, public contracts, or architectural ownership.
Acceptance Criteria
Post-Merge Validation
Out of Scope
- Diagnosing or curing the permanent 400% Ollama runner.
- Explaining why the vector database rejected a write.
- Cross-sweep non-convergence detection already delivered by #16875 / PR #16876.
- Provider timeout or saturation retry behavior tracked by #16012.
- Canary, admission, model-residency, or upstream Ollama cancellation policy.
Avoided Traps
- Do not present reduced duplicate work as proof of the 400% incident's cause.
- Do not cache a rejected provider promise or a partial/invalid result.
- Do not remove bounded write retries.
- Do not broaden this leaf into a pipeline or provider-policy rewrite.
Related
- Parent: #16780
- Prior detection-only leaf: #16875 / PR #16876
- Distinct provider-timeout retry mechanism: #16012
- Prototype branch:
ada/16780-embed-paid-once @ 9ea462ac02
Origin Session ID: ef6172cf-3843-4358-aa46-6275e47c9b87
Retrieval Hint: VectorService persistence retry re-buys identical embeddings upsert succeeds provider submissions paid once
Measured and prototyped by Ada (Claude Opus 5, Claude Code); leaf authored by Euclid (GPT-5.6, Codex Desktop).
Context
#16780 requires persistence failures to stop causing unbounded repeated embedding work. #16875 and merged PR #16876 made that non-convergence observable, but they explicitly left retaining vectors across a failed write out of scope.
Current
devstill placesTextEmbeddingService.embedTexts()andcollection.upsert()in the same retrytryinsideVectorService.embedChunks(). A transient vector-write failure therefore discards an already-successful embedding result and re-submits the identical texts to the provider on the next retry.A measured two-text / three-write-attempt falsifier produces 6 provider submissions on current
dev; retaining the successful result across write-only retries produces 2. The implementation prototype is onada/16780-embed-paid-onceat9ea462ac02cc91f9e963ba79fdeb97208a0dd69band must be rebased onto currentdevbefore PR creation.Memory mining surfaced the prior #16875 decision record (session
87f453f9-aa80-4487-9ed1-b5d91e052c43): detection was delivered, while survival across a failed write was deliberately deferred. A final duplicate sweep at 2026-08-11T11:44:26.702Z covered the 20 newest open GitHub issues, the 30 newest all-state A2A messages, and the local issue/discussion corpus; it found no separate live owner for this repair.This ticket does not claim to explain or cure the permanent 400% Ollama CPU incident. It removes a separately verified redundant-work multiplier.
The Problem
The retry boundary currently conflates two different failures:
Treating both as one attempt re-buys deterministic vectors, inflates provider activity, and can multiply expensive batch work by
maxRetrieswhile the collection makes no progress.Architectural Reality
textsToEmbedand ids are already stable for the batch.collection.upsert().??=preserves ordinary provider-failure retries.result.embeddedmust still advance only after persistence succeeds.VectorService.leaseYield.spec.mjsnegative control proves an ordinary embedding failure remains retryable.The Fix
embeddings ??= await TextEmbeddingService.embedTexts(...).Contract Ledger
VectorService.embedChunks()provider stepcollection.upsert()maxRetries, using the same ids, metadata, and vectors.result.embeddedDecision Record Impact
None. This narrows an existing retry boundary without changing configuration authority, public contracts, or architectural ownership.
Acceptance Criteria
embedded: 2.VectorServicesuites and the Knowledge Base unit directory pass on the rebased exact head.Post-Merge Validation
Out of Scope
Avoided Traps
Related
ada/16780-embed-paid-once@9ea462ac02Origin Session ID: ef6172cf-3843-4358-aa46-6275e47c9b87
Retrieval Hint:
VectorService persistence retry re-buys identical embeddings upsert succeeds provider submissions paid onceMeasured and prototyped by Ada (Claude Opus 5, Claude Code); leaf authored by Euclid (GPT-5.6, Codex Desktop).