Root-cause & characterize corrupted (empty-content) memories

Context

The operator flagged that the Memory Core holds a meaningful population of corrupted memories — AGENT_MEMORY entries whose core content (prompt / thought / response) is empty or missing. They are not cosmetic: a single session-summary run can take 30+ minutes grinding on them, and they distort backlog/cost accounting. Turn-/mini-summary generation was already hardened to skip corrupted memories (recent merged work, the #12628 / #12820 family), but session summarization still attempts to process them.

Live evidence (read-only probes, this session):

• memory-core-graph.sqlite Nodes: 3,333 AGENT_MEMORY rows with miniSummary IS NULL. Of those, 3,327 carry a semanticVectorId (Chroma pointer), 5 are bare shells, 177 belong to non-swarm/test identities, 62 lack a userId.
• SummarizationJobs: 432 completed / 32 failed / 6 in_progress — the stalls/give-ups the operator described.
• ai/services/memory-core/MemoryService.mjs:692 builds the embed text as [meta.prompt, meta.response].filter(Boolean).join(' — ') with no non-empty validation — an add_memory with empty content silently yields an empty-content memory + a meaningless vector.

V-B-A boundary (load-bearing): miniSummary IS NULL is a proxy, not the definition. The newest @neo-gpt nodes have full miniSummaries yet the operator reports recent gpt corruption — so the true predicate is empty/short prompt/thought/response, which lives in the Chroma neo-agent-memory document, not the SQLite node. The corrupted-set count must be measured against Chroma; that probe is deferred until the local embedder is idle (a heavy KB embedding run is active).

The Problem

Two coupled unknowns:

1. What is the corrupted set, exactly? Define "corrupted" operationally (empty / whitespace / below-min-length across prompt+thought+response), measure it in Chroma, cross-reference the graph nodes, and break down by agent + date — the operator specifically flagged recent @neo-gpt volume (gpt: 836 June AGENT_MEMORY nodes).
2. Why are they being written? Trace the call path invoking add_memory (or a direct graph write) with empty content. Candidates: a harness auto-save on low-content heartbeat/wake turns; a placeholder write; a save firing before prompt/thought/response are populated. The recent-gpt skew suggests a harness-/model-specific save path, not a universal one.

Until both are known we cannot safely delete (operator mandate: not a one-shot — know why + prevent recurrence first).

The Architectural Reality

• ai/services/memory-core/MemoryService.mjs:692 — [meta.prompt, meta.response].filter(Boolean).join(' — '); no mandatory / min-length guard on inputs (the write-gap).
• Raw content is the Chroma neo-agent-memory document, keyed by semanticVectorId; the SQLite Nodes row holds only metadata (name, stub description, agentIdentity, userId, sessionId, timestamp, miniSummary).
• ai/services/memory-core/SessionService.mjs summarizeSession — the path still processing corrupted memories (the 30-min stall); contrast the turn/mini-summary path that now skips them.
• SummarizationJobs (session_id PK, status, lease_token, expires_at, retry_count) — the stall/lease surface; in_progress past expires_at = stuck.
• Add-path entry: add_memory MCP tool → MemoryService.addMemory; plus any harness auto-persist path (see #10063).

The Investigation (deliverables, not a code fix)

1. Define the corrupted predicate (empty / whitespace / min-length over prompt+thought+response).
2. Read-only Chroma scan of neo-agent-memory for empty-content docs; count + agent/date breakdown; confirm or refute the recent-@neo-gpt concentration.
3. Trace the empty-content write source to a specific code/harness path with evidence.
4. Document the session-summary stall impact (which jobs, durations, lease/expiry behavior).
5. Hand off two follow-ups: the add_memory hardening (prevention) and the gated deletion.

Decision Record impact

none (no ADR asserted). Embodies the general "no silent empty/placeholder write" integrity principle.

Acceptance Criteria

• An operational "corrupted memory" predicate is defined and documented.
• The corrupted set is measured against Chroma (count + agent/date breakdown); the recent-@neo-gpt concentration is confirmed or refuted with numbers. (needs a clean embedder window)
• The empty-content write source is traced to a specific path (file:line or harness hook) with evidence.
• The session-summary stall impact is documented (affected jobs, durations, lease/expiry behavior).
• Findings handed off to the add_memory hardening ticket and the gated-deletion ticket (both linked).

Out of Scope

• Implementing the add_memory validation (separate prevention ticket).
• Deleting corrupted memories (separate, gated ticket — blocked on this + the hardening).
• The ask-tool synthesis timeout (separate; #12740 family).
• Test-isolation bleed (the 177 test-identity rows are a distinct concern, tracked under the test-bleed verify lane).

Avoided Traps

• Equating miniSummary IS NULL with "corrupted." Falsified: recent @neo-gpt nodes have miniSummaries yet are flagged corrupt; the predicate is empty raw content (Chroma), not the SQLite summary field.
• Deleting first. Operator mandate: root-cause + prevent before cleanup, or it recurs invisibly.
• Blaming the embedding run / model contention. This is a write-path defect, independent of the separately-tracked ask-tool latency.
• Folding in the test-bleed. The 177 test-identity rows are a distinct isolation concern, not the gpt corruption flagged here.

Related

• #12065 — [Epic] Orchestrator-as-SSOT for the REM (Sandman) Pipeline (parent; the pipeline that stalls on corrupted input)
• #12450 — query_summaries empty (sibling memory-integrity defect; different collection)
• #12743 — Gemini spend vs backfill volume (corrupted memories inflate backfill work)
• #12828 / #12829 — backfill scheduler spin on undrainable rows (mitigated the spin, not the cause)
• #10063 — Claude Code hook auto-persist turn memories (candidate write path)
• #12740 — local-first provider defaults (sibling epic for the prevention / ask-tool work)

Origin Session ID: 2feb15b9-1948-4553-9679-1419ed7eecf1

Handoff Retrieval Hints:

• Retrieval Hint: "corrupted empty-content memories add_memory no validation MemoryService 692 session summary stall"
• Proxy probe: SELECT COUNT(*) FROM Nodes WHERE json_extract(data,'$.label')='AGENT_MEMORY' AND json_extract(data,'$.properties.miniSummary') IS NULL → 3333; the real predicate = empty Chroma neo-agent-memory document.
• Log signature: session-summary runs exceeding 30 min; SummarizationJobs rows in_progress past expires_at.

Authored by Claude Opus 4.8 as @neo-opus-grace.