Audit 06-18 to 06-20 Memory Core vector-loss path

Context

The #13999 incident now has a pinned, public timeline but not yet a pinned culprit path.

Vega's #13999 comment (issuecomment-4804921485) walked backup bundle-meta.json manifests and empirically pinned the vector-loss window:

• last clean Memory Core export: 2026-06-18 07:42Z, 18,835 memories exported with retrievable vectors,
• first failed backup/export: 2026-06-20 05:23Z,
• current shape after the incident: metadata/doc rows remain materializable, but many Memory Core vector ids are absent from the persisted HNSW index.

That same comment falsified two prior theories: the May restore was still exporting cleanly through 2026-06-18, and the later embedding-cap fixes landed after the loss window. The still-open question is the in-window Chroma-mutating path: what ran between 2026-06-18 07:42Z and 2026-06-20 05:23Z that could turn present Memory Core vectors into metadata/doc rows without retrievable stored vectors?

This ticket is planning-mode capture only. It is intentionally unassigned and does not start implementation while the diagnostics/recovery gap set is still being mapped.

Pre-creation V-B-A:

• GitHub identity verified before public issue creation: neo-gpt.
• Label list verified: bug, ai, regression, architecture, and model-experience exist.
• Public #13999 comment read: https://github.com/neomjs/neo/issues/13999#issuecomment-4804921485 pins the loss window and names the remaining forensic audit.
• Public #13513 read: deterministic turn-aligned session chunker primitive shipped on 2026-06-19 as a safe split from #12073.
• Public #13551 read: embed-drain liveness watchdog shipped on 2026-06-20 as a detect/alarm follow-up for a silent drain-death incident.
• KB semantic sweep for Memory Core vector loss 2026-06-18 2026-06-20 REM tri-vector drain re-embed turn-aligned chunker embed-drain watchdog Chroma vectors surfaced #13999, #12073/#13513 lineage, #14000, and #10572, but no existing narrow culprit-path audit ticket.
• Local source scan identified current relevant surfaces: ai/services/graph/SemanticGraphExtractor.mjs, ai/daemons/orchestrator/services/DreamService.mjs, ai/services/memory-core/SessionService.mjs, and ai/daemons/orchestrator/scheduling/pipeline.mjs.
• Git commit-window scan over ai/services/graph, ai/services/memory-core, ai/daemons/embed, ai/daemons/orchestrator, ai/scripts/migrations, and ai/scripts/maintenance found relevant 2026-06-18→20 commits including #13513/#13514, #13551/#13553, #13469/#13470, #13471/#13472, #13503/#13559, #13601/#13603, #13634/#13635, #13583/#13584, and #13586/#13593.
• Live latest-open sweep: checked latest 25 open issues at 2026-06-25T23:15:07Z. Closest active tickets were #13999 (parent repair/exportability incident), #14024 (historical backup timeline diagnostic), #14026 (live data-integrity detect signal), and #14025 (test write guard). No equivalent culprit-path audit ticket was present.
• Exact live GitHub search for 06-18 06-20 vector loss audit REM returned only parent #13999.
• A2A in-flight sweep: checked latest 30 all-status messages at 2026-06-25T23:15:07Z. #14024/#14026/#14025 are planning/unclaimed or separate scopes; no competing claim for this forensic audit was present.

The Problem

We know when the data loss happened, but not how.

The dangerous unresolved mechanism is not generic "embedding was flaky." The failure shape is narrower: Memory Core documents and metadata still exist, while many stored vectors are missing from the vector index/export path. A correct forensic audit must distinguish these possibilities:

• a Chroma collection/segment mutation removed vectors while leaving metadata/doc rows,
• a repair/import/re-embed path reinserted metadata/docs without durable embeddings,
• a provider over-cap or provider-failure path persisted logical rows but failed vector creation,
• a maintenance path copied or promoted incomplete shadow state,
• an apparently related in-window change is exonerated by being read-only or detection-only.

Without that audit, follow-up work will keep treating symptoms: defrag repairs the corpus, #14024 dates the next corruption faster, #14026 alerts on future drift, and #14025 closes a test-store guard gap. None of those identifies the in-window producer that stripped or failed to create the vectors.

The Architectural Reality

Relevant public/code anchors:

• #13999 is the parent incident and contains the pinned backup-manifest timeline.
• #13513 / PR #13514 introduced the turn-aligned session chunker primitive during the suspect window. Current code computes Tri-Vector chunk budgets in ai/services/graph/SemanticGraphExtractor.mjs:293 and activates chunk-map/reduce at ai/services/graph/SemanticGraphExtractor.mjs:1036.
• #13551 / PR #13553 introduced the embed-drain liveness watchdog during the suspect window. Current code in ai/daemons/orchestrator/scheduling/pipeline.mjs:683 is read-only watchdog logic and should likely be exonerated, but it must be checked against the historical commit, not assumed.
• ai/daemons/orchestrator/services/DreamService.mjs:495 calls SemanticGraphExtractor.executeTriVectorExtraction(session) after Memory Session ingestion and records failure state at DreamService.mjs:515.
• ai/services/memory-core/SessionService.mjs:792 upserts session summaries into Chroma; SessionService.mjs:932 can upsert Antigravity plan artifacts into the memory collection. These are examples of logical-row Chroma mutation points that must be classified against the actual in-window execution paths.
• In-window maintenance commits also touched Chroma diagnostics/repair/export paths: #13469/#13470, #13471/#13472, #13503/#13559, #13601/#13603, #13634/#13635, and #13583/#13584.

The audit must compare the code as it existed in the 2026-06-18→20 window, not only current dev. Current code may already include repairs, guards, or exonerating comments that did not exist when the damage occurred.

The Fix

Perform a read-only forensic code audit and publish the result back to #13999 or a successor fix ticket:

1. Build a dated suspect matrix for every 2026-06-18 07:42Z→2026-06-20 05:23Z commit/process that could mutate Memory Core Chroma rows, vectors, collection identity, shadow promotion, import/export, or WAL/re-embed state.
2. For each suspect, classify mutation authority: read-only, metadata-only, graph-only, Chroma metadata/doc upsert, Chroma vector upsert, collection rename/promote/drop, export-only, or provider invocation.
3. Compare historical code at the relevant commit to current dev for each suspect. Do not infer from current comments alone.
4. Specifically audit the #13513/#13514 Tri-Vector chunking lineage, #13551/#13553 embed-drain watchdog lineage, and the in-window Chroma diagnostic/repair/export commits listed above.
5. Prove or falsify whether any path can persist Chroma metadata/doc rows without retrievable embeddings when the embedding provider rejects an oversized payload or is unavailable.
6. Produce a concise public result: culprit confirmed, suspect exonerated, or still-open with the next falsifier. If a concrete culprit is found, file the narrow repair/prevention ticket rather than widening #13999.

Contract Ledger Matrix

Decision Record impact

none. This is a forensic audit ticket under the existing #13999 incident. It may inform later ADR-0025/ADR-0026 work, but it does not itself amend diagnostics or actuator contracts.

Acceptance Criteria

• The audit produces a dated suspect matrix for the 2026-06-18 07:42Z→2026-06-20 05:23Z window.
• Every suspect is classified as read-only, metadata-only, graph-only, Chroma metadata/doc upsert, Chroma vector upsert, collection operation, export-only, provider invocation, or unknown-with-next-falsifier.
• #13513/#13514 and #13551/#13553 are explicitly audited and either implicated or exonerated with evidence.
• In-window Chroma diagnostics/repair/export commits (#13469/#13470, #13471/#13472, #13503/#13559, #13601/#13603, #13634/#13635, #13583/#13584) are included in the matrix.
• The audit proves or falsifies whether an oversized/provider-failure path could persist Memory Core metadata/docs without retrievable vectors in the audited code path.
• The result is posted publicly on #13999 or turned into a narrow successor repair/prevention ticket with a truthful close target.
• No live Memory Core mutation is performed as part of the audit.

Out of Scope

• Running live defrag or repair.
• Implementing the #14024 backup timeline diagnostic.
• Implementing the #14026 live data-integrity detect-signal.
• Implementing the #14025 Chroma test write guard.
• Reopening the May-restore theory unless new evidence contradicts the backup-manifest timeline.
• Blaming embedding caps that landed after the loss window.

Avoided Traps

• Do not treat current dev as proof of historical behavior; compare historical commits.
• Do not collapse "provider was flaky" into a root cause without showing the persistence boundary that leaves metadata/docs without vectors.
• Do not mix this with live repair. Repair can proceed operationally, but this ticket is the cause audit.
• Do not assign implementation ownership while planning mode remains active.

Related

Related: #13999 Related: #14024 Related: #14026 Related: #14025 Related: #13513 Related: #13551 Related: #13469 Related: #13471 Related: #13503 Related: #13601 Related: #13634 Related: #13583

Origin Session ID: 35f83031-f1a6-41a7-9c3b-089b87307db9

Handoff Retrieval Hints

• query_raw_memories("06-18 06-20 Memory Core vector loss REM tri-vector drain re-embed culprit path")
• query_raw_memories("#13999 backup manifest timeline in-window Chroma-mutating path")
• Exact anchors: issuecomment-4804921485, git log --since='2026-06-18T00:00:00Z' --until='2026-06-20T23:59:59Z', SemanticGraphExtractor.executeTriVectorExtraction, DreamService, SessionService.summarizeSession, embed-drain-liveness-watchdog, defragChromaDB, checkChromaIntegrity.