Log Memory Core repair progress during defrag

Context

During the live Memory Core recovery work, the operator ran the repair-defrag flow from a separate visible terminal and saw only:

Initial Size: 1716.15 MB
Creating pre-nuke snapshot...
Connecting to ChromaDB...
Memory Core repair-defrag: full-enumeration extract + re-embed + shadow-promote...

Then the script emitted no progress while operating on a badly bloated Chroma store. With this level of corruption, the repair can plausibly run for hours, so a silent extraction/re-embed phase is operationally unsafe: the operator cannot distinguish a slow repair from a hung one.

Live latest-open sweep: checked the latest 20 open issues at 2026-06-25T15:23:22Z; no equivalent progress-logging ticket found. A2A in-flight sweep: checked latest 30 messages at 2026-06-25T15:23:22Z; no overlapping lane claim found. Label check: confirmed bug, ai, regression, and model-experience exist before filing.

Release classification: boardless post-v13 incident-stability slice. Project 12 is the closed v13 board and must not receive new items. Creation correction: this issue was initially attached to Project 12 because the local ticket-create rule still named the v13 board after release closeout; the Project 12 membership has been removed.

The Problem

The repair-defrag path performs three expensive operations: full metadata/vector coverage enumeration, row extraction / missing-vector re-embed, and shadow promotion. The current operator-visible log marks the start of the repair but does not expose progress inside the long-running phases. That creates a false operational dead zone exactly when the system is degraded and human observation matters most.

A second adjacent hazard surfaced while implementing the progress slice: stale duplicate collection-name rows can exist in Chroma. If the repair code picks the first matching coverage row by name instead of the live collection id, the progress and repair path can report or act on the wrong backing collection.

The Architectural Reality

The affected substrate is the maintenance-script path, not the Memory Core MCP API:

• ai/scripts/maintenance/defragChromaDB.mjs owns the operator-facing defrag flow and shadow-promotion orchestration.
• ai/scripts/maintenance/repairMemoryCoreStoredEmbeddings.mjs owns full-enumeration row extraction and missing-vector re-embed.
• Chroma can retain stale rows for the same logical collection name, so coverage selection must be disambiguated by the live collection id before extraction or promotion.

The Fix

Add operator-visible progress logging to the repair-defrag path:

• log coverage enumeration start and completion;
• log per-collection metadata/vector/missing/extra counts before repair work;
• emit 10%-bucket progress during intact-vector extraction;
• emit 10%-bucket progress during missing-vector re-embed;
• log shadow promotion start and completion;
• guard duplicate collection-name coverage rows by selecting the row whose id matches the live Chroma collection id, and fail loud if no live-id match exists.

Contract Ledger Matrix

Decision Record impact

none - this is operational observability and guardrail work inside the existing repair-defrag path.

Acceptance Criteria

• The repair-defrag script logs a coverage-enumeration start and completion message before per-collection repair.
• Each Memory Core collection logs metadata/vector/missing/extra coverage counts before extraction.
• Intact-vector extraction emits progress at least every 10% for long collections.
• Missing-vector re-embed emits progress at least every 10% for long collections.
• Shadow promotion logs start and completion for each repaired collection.
• Duplicate logical collection-name coverage rows are resolved by live collection id, not first-row order.
• The code fails loud before extraction/promotion when no coverage row matches the live collection id.
• Unit coverage proves the progress callback and duplicate-row guard behavior.

Out of Scope

• Completing the full Memory Core backup/export parity fix tracked by #13999.
• Reopening or changing already-merged repair PRs.
• Changing embedding batch size, provider selection, or canonical backup format.
• Running the operator's live repair terminal from an agent-owned hidden process.
• Fixing stale test-* Chroma collection reappearance, tracked separately in #14010.

Avoided Traps

• Do not treat the progress log as proof that the broader repair succeeded. It only makes the repair observable.
• Do not auto-close #13999 from this slice. The broader incident still needs canonical backup parity and live recovery evidence.
• Do not select Chroma coverage rows by collection name alone when duplicate stale rows exist.

Related

#13999 #14010

Origin Session ID: 9280140f-8b54-4462-9342-49cca7e226f4

Retrieval Hint: Memory Core repair-defrag progress 10%-bucket logs duplicate collection id guard