Restore Memory Core backup exportability after partial export

Context

The orchestrator backup attempt failed on 2026-06-25 at 10:14:46Z while running the Agent OS backup. The failure was captured in ProcessSupervisor stderr:

DATABASE_EXPORT_ERROR: PARTIAL_COLLECTION_EXPORT: undefined exported 0/22531 records to .../mc/memory-backup-2026-06-25T10-14-45.557Z.jsonl; skipped 2000 corrupted vector id(s).

The stack points at ai/services/memory-core/DatabaseService.mjs:415, called from ai/scripts/maintenance/backup.mjs:188, under withHeavyMaintenanceLease(...) in ai/daemons/orchestrator/services/HeavyMaintenanceLeaseService.mjs:599.

Health verification after the harness restart:

• Memory Core MCP healthcheck: status: healthy, runtime freshness current, neo-agent-memory count 22531, neo-agent-sessions count 1409, identity bound to @neo-gpt.
• Knowledge Base MCP healthcheck: status: healthy, runtime freshness current, neo-knowledge-base count 32277.
• Chroma heartbeat responded on localhost:8000.
• Memory Core health reports backup.lastSuccessful: 2026-06-18T07-42-28.523Z.

Prior-art sweep:

• #13496 captured the broader Memory Core stored-embedding exportability repair lane and is closed.
• #13583 / PR #13584 made partial Memory Core Chroma exports fail loudly instead of silently producing false-complete backups, and is closed.
• The current incident is therefore not "fail loud is missing"; the fail-loud contract is working. The unresolved P0 is that the live Memory Core collection is still not exportable enough for the canonical backup to complete.

Live latest-open sweep: checked the latest 20 open GitHub issues immediately before filing on 2026-06-25; no equivalent open ticket for this P0 backup-exportability failure was present.

A2A in-flight claim sweep: checked latest 30 all-status A2A messages immediately before filing on 2026-06-25; no competing [lane-claim] / [lane-intent] for this Memory Core backup regression was present.

Release classification: ON Project 12. Operator classified the incident as priority 0; backup integrity is release/runtime safety, and the active backup path cannot currently complete for Memory Core.

The Problem

The canonical Agent OS backup now correctly refuses to treat a partial Memory Core export as complete, but the production collection state still causes the export path to fail:

• expected records: 22531
• exported records: 0
• skipped corrupted vector ids: 2000
• reported collection: undefined
• failed artifact path: .../mc/memory-backup-2026-06-25T10-14-45.557Z.jsonl

This means the safety mechanism from #13583 is doing the right thing, but the operational state remains unsafe: the current Memory Core data cannot be backed up through the canonical logical JSONL backup path.

Two details matter for follow-up:

1. The issue is not visible from ordinary MC/KB health alone. Both MCP healthchecks are healthy after restart, and Chroma answers heartbeat. Exportability must remain a separate axis from queryability and liveness.
2. The error reports collection: undefined, which weakens operator triage. The repair should preserve or reconstruct the collection identity in the error path even when the Chroma collection object does not expose .name.

The Architectural Reality

Relevant surfaces:

• ai/services/memory-core/DatabaseService.mjs owns Memory Core JSONL export/import. #exportCollection() counts, paginates, rescues per id, tracks skipped ids, and throws PARTIAL_COLLECTION_EXPORT when exported !== expected.
• ai/scripts/maintenance/backup.mjs is the canonical atomic backup bundle orchestrator. It exports KB first, then Memory Core memories/summaries, then graph, then verifies row-count parity before writing bundle-meta.json.
• ai/daemons/orchestrator/services/HeavyMaintenanceLeaseService.mjs runs the backup under the heavy-maintenance lease, so this failure shows up as an orchestrator-managed incident rather than an isolated manual script failure.
• #13496 established that Memory Core query health and stored-embedding exportability are distinct. #13583 intentionally kept live Chroma repair/rebuild out of scope.

The Fix

Investigate and repair the Memory Core exportability state without weakening the fail-loud backup contract:

1. Reproduce the failing exportability class with a bounded, non-mutating diagnostic against the affected Memory Core collection(s), including enough sample ids to explain why the first rescue batch exports 0/2000.
2. Identify whether the failure is a collection-wide stored-embedding materialization break, a provider/vector-dimension incompatibility, a Chroma metadata/vector index divergence, or a recent embedding-ingest regression.
3. Select a safe repair path for current production data. Candidate shapes include re-embedding from documents/metadata through service boundaries, rebuilding shadow collections before swap, or restoring from a known-good backup then replaying missing WAL/graph state.
4. Keep PARTIAL_COLLECTION_EXPORT fail-loud behavior intact. Do not make the backup pass by suppressing corrupted ids or marking an incomplete backup as complete.
5. Improve export error diagnostics so collection is never undefined in the thrown error details or human log line.
6. Add focused regression coverage for the new repair/diagnostic behavior and the non-undefined collection identity in partial-export errors.

Contract Ledger Matrix

Decision Record impact

none. This stays within ADR 0017's unified Chroma topology and preserves the #13583 fail-loud backup contract. It is an operational data-integrity repair, not a topology change.

Acceptance Criteria

• A bounded, non-mutating diagnostic reproduces the current Memory Core exportability failure and identifies the affected collection role/name.
• The investigation explains why the current export path reports exported: 0, expected: 22531, and skipped: 2000.
• A copy/shadow validated repair path is selected before any live collection mutation.
• After repair, the canonical backup path completes Memory Core export with row-count parity.
• PARTIAL_COLLECTION_EXPORT remains fail-loud; incomplete backups cannot be reported as successful.
• Partial-export errors and logs never report collection: undefined.
• Focused tests or scripted verification cover the repaired diagnostic/error path.

Out of Scope

• Weakening or reverting #13583's fail-loud behavior.
• Treating MC/KB healthcheck success as proof of backup exportability.
• Live Chroma delete/recreate, REINDEX, VACUUM, or collection swap without copy/shadow validation and operator-safe sequencing.
• Redesigning backup retention or the heavy-maintenance lease scheduler.

Avoided Traps

• Do not "fix" the P0 by letting the backup skip corrupted ids and continue.
• Do not conflate queryability with exportability; the restarted MCP healthchecks are healthy while backup export still failed.
• Do not assume the FTS5 repair class fixes stored-vector materialization.
• Do not bury the useful production evidence behind a generic DATABASE_EXPORT_ERROR; preserve expected/exported/skipped/collection details.

Related

Related: #13496 Related: #13583 Related: #13584 Related: #13946

Origin Session ID: a49576d1-b873-496e-b4b9-efaf44b05a24

Handoff Retrieval Hints

• query_raw_memories("Memory Core backup failed PARTIAL_COLLECTION_EXPORT corrupted vector ids")
• query_raw_memories("Memory Core Chroma partial export skipped corrupted vector ID backup completeness")
• Exact anchors: ai/services/memory-core/DatabaseService.mjs, ai/scripts/maintenance/backup.mjs, ai/daemons/orchestrator/services/HeavyMaintenanceLeaseService.mjs