LearnNewsExamplesServices
Frontmatter
id14218
titleDe-dup slice-4: route the embed-repair re-embed through resolveTurnDocumentForRead before the document drop
stateClosed
labels
enhancementaiarchitecture
assigneesneo-opus-vega
createdAtJun 27, 2026, 8:59 AM
updatedAtJun 27, 2026, 6:07 PM
githubUrlhttps://github.com/neomjs/neo/issues/14218
authorneo-opus-vega
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJun 27, 2026, 6:07 PM

De-dup slice-4: route the embed-repair re-embed through resolveTurnDocumentForRead before the document drop

Closed v13.1.0/archive-v13-1-0-chunk-7 enhancementaiarchitecture
neo-opus-vega
neo-opus-vega commented on Jun 27, 2026, 8:59 AM

Problem

The third turn-document reader from the #14193 slice-4 cross-reader audit — the sharpest one (the circularity). ai/scripts/maintenance/repairMemoryCoreStoredEmbeddings.mjs re-embeds missing-vector rows FROM their stored Chroma document (got.documents?.[j], ~L140). Once slice-4 drops the redundant stored turn-document, a de-duped turn's missing-vector row reads a null/empty document → getDocumentProblem records it unrecoverable — even though it is fully reconstructable from the split metadata fields. So the embed-repair safety-net (the path that REGENERATES a missing vector) would fail to recover exactly the de-duped turns it exists to fix.

This is the maintenance-half reader; #14211 (#14213) covers the runtime readers (DreamService / SessionService) and is scoped to them, so this needs its own leaf. Ada handed the call to me/Grace.

Fix shape

Route the missing-vector re-embed's document read through resolveTurnDocumentForRead (#14210): reconstruct a dropped turn-document from its split metadata (the read already includes metadatas), then re-embed the reconstructed text. The reconstruct is byte-identical to the original (composeTurnDocumentText), so the regenerated vector is correct. The INTACT-rows path (L106) stays as-is — those rows keep their existing vectors; an empty stored doc there is de-dup-consistent (the document is intentionally dropped).

Acceptance Criteria

  • repairMemoryCoreStoredEmbeddings reconstructs a dropped turn-document via resolveTurnDocumentForRead before re-embedding, so a de-duped turn's missing vector is recovered, not marked unrecoverable.
  • A genuinely-absent / non-turn document still records unrecoverable (no false recovery).
  • Unit coverage for the dropped-document → reconstruct → re-embed path.
  • Lands before slice-4 drops any stored turn-document.

Scope

v13.1 — slice-4 reader-adoption (maintenance-half). Parent #14193 (de-dup epic); depends on #14210 (the resolver). Surfaced by the #14193 cross-reader audit.

Live latest-open sweep: scanned the de-dup cluster (#14193 / #14211 / #14209 / #14213 / #14210) and searched repairMemoryCoreStoredEmbeddings / re-embed-reconstruct — no existing leaf for the maintenance-half reader. No duplicate.

Authored by Vega (Claude Opus 4.8, Claude Code). Origin Session ID: 1bb8a27b-ae0d-4668-a9a2-acbbe2387512

tobiu referenced in commit 82255a3 - "fix(ai): reconstruct dropped turn-documents in the embed-repair re-embed — recover de-duped turns (#14218) (#14219) on Jun 27, 2026, 6:07 PM
tobiu closed this issue on Jun 27, 2026, 6:07 PM