LearnNewsExamplesServices
Frontmatter
id13093
titleGate corrupted-memory cleanup behind provenance-safe classification
stateClosed
labels
bugaiarchitecturemodel-experience
assigneesneo-gpt
createdAtJun 13, 2026, 1:42 PM
updatedAtJun 13, 2026, 4:58 PM
githubUrlhttps://github.com/neomjs/neo/issues/13093
authorneo-gpt
commentsCount2
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 13, 2026, 4:58 PM

Gate corrupted-memory cleanup behind provenance-safe classification

Closed v13.1.0/archive-v13-1-0-chunk-2 bugaiarchitecturemodel-experience
neo-gpt
neo-gpt commented on Jun 13, 2026, 1:42 PM

Context

#12830 now has the missing AC#2 characterization addendum from 2026-06-13: the live Chroma neo-agent-memory metadata scan measured 18,491 rows and found 74 rows with at least one empty payload field. The split matters: 48 rows have prompt + thought + response empty, while 26 rows have only prompt empty.

This ticket exists because the measurement closes the characterization gap, but deletion remains a separate destructive act. The operator mandate on this lane is preservation-first: do not delete potentially valuable memory rows just because a predicate matched them.

Release classification: post-release hardening cleanup; boardless. This follows from #12830 being characterized as non-cut-blocking, and from the destructive nature of cleanup requiring its own gate.

Live latest-open sweep: checked latest 20 open issues at 2026-06-13T11:41:06Z; no equivalent cleanup/deletion-gate ticket found. Exact open-issue searches for gated deletion corrupted memory empty-content and AGENT_MEMORY cleanup prompt-only-empty found #12830 only. A2A in-flight sweep: checked the latest 30 inbox messages at 2026-06-13T11:41Z; no recent [lane-claim] / [lane-intent] overlapped this cleanup scope. Knowledge Base semantic sweep was attempted and failed because the KB collection was unavailable; local issue/discussion exact scan found #12830 and #12879, but no separate cleanup ticket.

The Problem

The corrupt set is small enough to clean, but the provenance is weak enough to make direct deletion unsafe:

  • 74 Chroma rows match the AC#2 predicate where at least one of metadata.prompt, metadata.thought, or metadata.response is non-string or whitespace/empty.
  • 48 are strict all-fields-empty candidates.
  • 26 are prompt-only-empty rows, which may still preserve useful thought/response content and must not be collapsed into the all-empty class.
  • Only 27 of the 74 rows matched graph Nodes by id during the addendum cross-reference.
  • 0 rows resolved to known Chroma or graph agent provenance in that scan.

Without a gated cleanup procedure, the obvious failure mode is a broad delete keyed to an overly coarse predicate. That would convert a characterization win into irreversible data loss.

The Architectural Reality

The source of truth for this cleanup predicate is Chroma neo-agent-memory metadata fields, not SQLite miniSummary IS NULL, not the Chroma document text, and not a document label. #12830 already falsified miniSummary IS NULL as the corruption definition, and Ada's earlier investigation noted that the embedded document can remain non-empty even when the payload fields are empty.

#12879 is the prevention/hardening ticket and was live-checked as closed on 2026-06-13. This ticket should not re-open the write-path guard; it should consume the guard as a prerequisite and verify no new corrupt rows appeared after the AC#2 scan window before any cleanup mutation.

The cleanup consumer is both human and agentic: reviewers need a compact manifest and dry-run evidence before approving any destructive action, and future agents need a repeatable predicate that does not drift into deleting prompt-only or provenance-unknown rows by accident.

The Fix

Define and execute a preservation-first cleanup gate for the measured #12830 corrupt set:

  1. Re-run the read-only Chroma metadata scan and graph cross-reference to refresh the 74-row candidate set.
  2. Classify candidates into at least two explicit classes: all-fields-empty and prompt-only-empty.
  3. Produce a redacted review manifest containing ids, class, timestamp/month, session id when present, graph-match status, and provenance status. Do not publish raw prompt/thought/response content in a public artifact.
  4. Run a dry-run cleanup command/report that emits counts and target ids only, with zero mutation.
  5. Require explicit operator approval, with peer-visible evidence, before any destructive mutation.
  6. Prefer archive/quarantine or reversible backup before permanent deletion; if deletion is chosen, prove the backup/snapshot exists first.
  7. Post-cleanup validation must show the chosen strict predicate is gone while unrelated Memory Core collections remain intact.

No new .mjs file path is prescribed here. If the implementing agent decides a new maintenance script is needed, they must run structural pre-flight at intake before choosing the directory.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs / Evidence
Cleanup candidate predicate #12830 AC#2 addendum Match Chroma neo-agent-memory metadata payload fields; split all-empty from prompt-only-empty Halt if Chroma scan cannot be reproduced Issue comment on #12830, refreshed dry-run output
Review manifest This ticket Redacted ids + classification + provenance, no raw payload content in public Keep ticket blocked until manifest exists Manifest summary in issue/PR body or private operator handoff if content sensitivity appears
Destructive mutation gate Operator authority + peer-visible evidence No deletion without explicit operator approval after dry-run and backup/quarantine evidence Quarantine/archive only, or no-op Approval comment plus validation report
Post-cleanup validation Memory Core collection integrity Strict selected predicate returns zero for approved target class; unrelated collection counts are stable Restore from backup/snapshot Validation output attached to cleanup PR/comment

Decision Record impact

none (cleanup procedure for an already-characterized Memory Core integrity defect; no ADR amendment asserted).

Acceptance Criteria

  • Re-run a read-only Chroma metadata scan of neo-agent-memory and graph cross-reference, producing candidate counts for the current state.
  • Classify candidates separately as all-fields-empty vs prompt-only-empty; do not delete prompt-only-empty rows under the all-empty predicate.
  • Produce a redacted manifest for reviewer/operator inspection before mutation, including ids, class, timestamp/month, session id when present, graph-match status, and provenance status.
  • Verify prevention is in place before cleanup: #12879 is closed/landed and a fresh scan shows no new corrupt rows after the 2026-06-13 AC#2 addendum window.
  • Provide a dry-run cleanup report that performs no mutation and lists exact target ids/counts for the proposed action.
  • Obtain explicit operator approval before any destructive deletion; peer approval alone is not enough for deletion.
  • If deletion is approved, create backup/quarantine evidence before mutation and document how rollback would work.
  • After cleanup, validate that the approved strict predicate is gone and unrelated Memory Core collections/counts were not damaged.

Out of Scope

  • Changing the add_memory write-path validation; that was tracked by #12879.
  • Session-summary timeout/abort-guard work.
  • Using miniSummary IS NULL as a deletion predicate.
  • Deleting prompt-only-empty rows without separate review and explicit approval.
  • Publishing raw memory payload content in a public issue, PR, or comment.

Avoided Traps

  • Delete first. This ticket exists specifically to prevent one-shot deletion from replacing root-cause work.
  • Treat 74 as one homogeneous class. The 48 all-empty rows and 26 prompt-only-empty rows have different preservation risk.
  • Treat unknown provenance as safe deletion. Unknown provenance raises the review bar; it does not lower it.
  • Use SQLite miniSummary IS NULL as the target set. That predicate was already rejected by #12830.
  • Assume the guard makes old cleanup safe. Prevention stops recurrence; it does not prove historical rows are valueless.

Related

  • #12830 — root-cause and characterize corrupted empty-content memories; AC#2 addendum posted on 2026-06-13.
  • #12879 — prevention/hardening path for add_memory empty-content writes; live-checked closed on 2026-06-13.
  • #12065 — REM/session-summary pipeline context affected by corrupted memory input.
  • #12829 — memory-summary backfill scheduler mitigation, related preservation-first precedent.

Origin Session ID: d2d31447-5009-47a8-992e-9ecc35b806c1

Handoff Retrieval Hints:

  • Retrieval Hint: "#12830 AC#2 Chroma corrupt set 74 rows 48 all-empty 26 prompt-only-empty gated deletion cleanup"
  • Retrieval Hint: #12830 comment IC_kwDODSospM8AAAABGAv5yA contains the measurement addendum and exact count split.