Extraction-with-re-embed primitive — the first slice of #13496's AC4 repair (the novel half). #13496's diagnostics are done (#13501/#13502 exportability probe, #13584 fail-loud partial export, #13503/#13559 vector-coverage audit). The measured failure: Memory Core collections have metadata/vector-index divergence (neo-agent-memory: ~13,917 of 18,848 vectors missing); collection.get({include:['embeddings']}) fails Error finding id for the missing ids.
The Architectural Reality
defragChromaDB's shadow/parking promotion is disabled for MC because it extracts via include:['embeddings'], which fails for the missing-vector ids. This leaf supplies the missing extraction half so defrag's existing addCollectionData → validateLoadedCollection → promote can be reused: partition each collection's ids into intact-vector vs missing-vector, extract intact rows with their stored vectors, and re-embed the missing-vector rows from their documents (which still materialize).
Acceptance Criteria (extraction primitive only)
extractMemoryCoreCollectionData({collection, allIds, missingVectorIds, embedFn}) → {data, unrecoverable, counts}: partitions intact vs missing, extracts intact rows with stored vectors, re-embeds missing-vector rows from documents (via embedFn), and surfaces document-less / metadata-absent rows as unrecoverable with counts (fail-loud, never silently dropped).
The live shadow run + canonical promotion is operator/env-gated (the sandbox can't reach Chroma — IPv6-loopback only; live mutation needs explicit operator authorization).
Context
Extraction-with-re-embed primitive — the first slice of #13496's AC4 repair (the novel half). #13496's diagnostics are done (#13501/#13502 exportability probe, #13584 fail-loud partial export, #13503/#13559 vector-coverage audit). The measured failure: Memory Core collections have metadata/vector-index divergence (neo-agent-memory: ~13,917 of 18,848 vectors missing);
collection.get({include:['embeddings']})failsError finding idfor the missing ids.The Architectural Reality
defragChromaDB's shadow/parking promotion is disabled for MC because it extracts viainclude:['embeddings'], which fails for the missing-vector ids. This leaf supplies the missing extraction half so defrag's existingaddCollectionData → validateLoadedCollection → promotecan be reused: partition each collection's ids into intact-vector vs missing-vector, extract intact rows with their stored vectors, and re-embed the missing-vector rows from their documents (which still materialize).Acceptance Criteria (extraction primitive only)
extractMemoryCoreCollectionData({collection, allIds, missingVectorIds, embedFn})→{data, unrecoverable, counts}: partitions intact vs missing, extracts intact rows with stored vectors, re-embeds missing-vector rows from documents (viaembedFn), and surfaces document-less / metadata-absent rows asunrecoverablewith counts (fail-loud, never silently dropped).embedTexts): intact-only (embedFn not called); mixed (intact keep stored, missing re-embedded); document-less →unrecoverable; metadata-absent →unrecoverable;embedFnwrong-length → throws; missingembedFn→ throws.Next slices (remain on parent #13496, NOT this leaf)
These were narrowed OUT of this leaf (per @neo-gpt's #13603 close-target review) and stay on parent #13496 as the remaining AC4 repair work:
Related
Related: #13496 Related: #13584 Related: #12450
Origin Session ID: abe80be3-6235-4a9e-99bc-b14659ba806a