Surface unrecoverable reasons in Memory Core defrag

Context

Vega approved PR #14021 with a non-blocking follow-up after reviewing the defrag resumability and heavy-maintenance lease work:

• Review: https://github.com/neomjs/neo/pull/14021#pullrequestreview-4575319345
• PR state verified live on 2026-06-25T22:07Z: #14021 is merged, review decision APPROVED, and all listed checks are successful.

The follow-up is not a reopen of #14021. The merged fix is structurally correct and unblocks the live Memory Core repair path, but the operator-facing unrecoverable-row report still lacks enough cause metadata for fast triage during a long repair run.

Pre-creation V-B-A:

• Source read: ai/scripts/maintenance/repairMemoryCoreStoredEmbeddings.mjs currently returns unrecoverable: String[] from extractMemoryCoreCollectionData() and increments counts for missing metadata/document rows.
• Source read: ai/scripts/maintenance/defragChromaDB.mjs currently logs unrecoverable rows as counts and ids only, for example ABORTED ... unrecoverable row(s); counts ....
• KB ticket sweep: semantic search found #13999 / #14005 as the parent repair/exportability context but no existing narrow ticket for per-row unrecoverable cause metadata.
• GitHub live latest-open sweep: checked the latest 20 open issues at 2026-06-25T22:09:04Z; no equivalent follow-up was present.
• A2A in-flight sweep: checked the latest 30 all-status A2A messages at 2026-06-25T22:09:04Z; no competing lane claim or lane intent for this specific follow-up was present.

The Problem

The Memory Core repair-defrag path now preserves progress and fails loud when unrecoverable rows remain, but the failure payload is too thin for operator triage:

• the count is fail-loud,
• the ids are available,
• the reason is not preserved at the row level.

That means a live repair abort can tell the operator that rows are unrecoverable, but not whether each row failed because the metadata read omitted the id, the document was empty/missing, the provider rejected an over-cap document, the provider was down, or the provider returned malformed embedding output.

For the current Agent OS stability incident, this matters because a repair can run for hours. If it aborts near the end, the operator should not need a second manual investigation pass to classify the failed ids.

The Architectural Reality

Relevant surfaces:

• ai/scripts/maintenance/repairMemoryCoreStoredEmbeddings.mjs owns Memory Core missing-vector extraction and re-embedding. Its JSDoc already promises fail-loud unrecoverable reporting, but the current contract returns only unrecoverable: String[].
• ai/scripts/maintenance/defragChromaDB.mjs owns the operator-facing Memory Core repair-defrag CLI output. It currently logs counts and prints the unrecoverable count in final abort lines.
• PR #14021 added durable resumability and lease adoption. This follow-up must preserve that repair contract and only enrich the diagnostic payload.
• Tests under test/playwright/unit/ai/scripts/maintenance/ already cover unrecoverable rows, dry-run abort reporting, and Memory Core repair CLI behavior.

The Fix

Enrich the Memory Core repair unrecoverable report with row-level reason metadata:

1. Change the internal unrecoverable shape from bare ids to structured entries such as {id, reason, message?} while preserving a compatibility path for callers/tests that only need ids, if needed.
2. Capture distinct reasons at the extraction boundary:
   • metadata/document row absent from the Chroma get({include:['documents','metadatas']}) response,
   • empty/missing document,
   • per-document embedding failure after batch isolation,
   • malformed provider return shape where it can be attributed without losing fail-loud behavior.
3. Surface those reasons in the dry-run and mutating repair abort output, including a bounded preview suitable for terminal logs.
4. Preserve fail-loud semantics: any unrecoverable entry still aborts promotion and exits non-zero at the CLI boundary.
5. Optional edge polish if it remains small and local: detect when all per-doc isolation attempts fail in a batch and emit a likely provider-down/provider-unavailable reason instead of logging thousands of indistinguishable row failures; wrap the missing resumable-shadow error with a clearer operator message.

Contract Ledger Matrix

Decision Record impact

none. This is a diagnostic contract enrichment inside the existing Memory Core repair-defrag path. It does not change ADR 0017 unified Chroma topology or the heavy-maintenance lease contract from #14021.

Acceptance Criteria

• extractMemoryCoreCollectionData() records row-level unrecoverable reasons instead of only bare ids.
• Missing metadata/document rows and empty/missing documents produce distinct reason values.
• Per-document embedding failures after batch isolation preserve a useful reason/message without retrying indefinitely.
• Memory Core repair dry-run and mutating abort logs include a bounded unrecoverable reason preview.
• Promotion remains fail-loud: any unrecoverable entry still blocks shadow promotion and returns a failed repair result.
• Focused unit coverage verifies the new unrecoverable reason contract and updated operator-facing output.

Out of Scope

• Reopening #14021.
• Changing the heavy-maintenance lease behavior.
• Changing the shadow/park/promote/rollback contract.
• Running a live mutating defrag in CI or unit tests.
• Solving every provider outage behavior; only classify/report it when the local isolation path can do so cheaply.

Avoided Traps

• Do not turn this into another repair-defrag rewrite. The merged shape is correct; this ticket is diagnostic enrichment.
• Do not weaken fail-loud behavior to make the terminal output look cleaner.
• Do not log unbounded thousands of row details by default; use a bounded preview plus counts.
• Do not hide provider-level failures behind the same label as genuinely bad rows when the distinction is available.

Related

Related: #13999 Related: #14020 Related: #14021

Origin Session ID: 019efe4c-5d55-76c0-aba5-665f86d9cbdc

Handoff Retrieval Hints

• query_raw_memories("PR 14021 Vega Approve Follow-Up unrecoverablePreview embedRecoverableDocuments failure reason")
• query_raw_memories("Memory Core repair-defrag unrecoverable reason provider down missing document")
• Exact anchors: ai/scripts/maintenance/repairMemoryCoreStoredEmbeddings.mjs, ai/scripts/maintenance/defragChromaDB.mjs, test/playwright/unit/ai/scripts/maintenance/repairMemoryCoreStoredEmbeddings.spec.mjs, test/playwright/unit/ai/scripts/maintenance/defragMemoryCoreRepair.spec.mjs