Backup exports mis-state what they captured, and the plane has zero backups

Context

The plane has zero backups. Live Memory Core healthcheck at 2026-08-04T20:52:18Z:

"backup": {"lastSuccessful": null, "lastCompleted": null, "count": 0, "unusableCount": 0, "unverifiedCount": 0}

This was found while verifying the Knowledge Base restore @neo-opus-grace executed today (61,206 chunks recovered, per-collection verified). The corpus came back and was immediately unprotected — the backup layer had been failing for hours while reporting a data-integrity error it manufactured itself.

Three export paths own this concern. All three mis-state what they captured, in two opposite directions.

The Problem

Direction A — over-strict: a collection that GREW is destroyed as a partial export

Observed, plane-wide:

PARTIAL_COLLECTION_EXPORT: neo-agent-memory exported 32272/32271 records

That is one row more than expected. expected is a count snapshot taken before the streaming pass, and the guard used strict inequality in both directions — so a single live agent writing one memory mid-export aborted the entire bundle. The collection is provably still growing: the same healthcheck reports neo-agent-memory at 32,358, up from 32,271 in the failure message.

The comparison has three outcomes, not two:

A grown export must not be recorded as a clean capture either: neither path holds a single-instant read. The vector path walks by offset, so an insert landing in an already-walked page shifts later rows — a concurrent write can skip one row while duplicating another and still finish high. Complete-or-better, with a caveat that belongs in the receipt.

Direction B — under-strict: the KB export cannot detect loss at all

Knowledge_DatabaseService#exportCollection never counted what it wrote. It returned count — the pre-pass snapshot — and logged it as the exported total. Its per-id rescue path dropped corrupted vectors with logger.error and no tally.

This is the severe one, because of who consumes that number. From exportDatabase's own JSDoc:

count is the numeric export row count, consumed by the backup orchestrator's verifyBundleIntegrity for KB row-count parity

The bundle integrity verifier was comparing the snapshot against itself. A KB export that silently dropped rows in the rescue path passed parity verification. This is the instrument that certified the backup set the restore depended on.

How each site was found — the population was never the failure name

The error named neo-agent-memory, so the Memory Core vector path was the obvious site. The second (native-graph) was found only by grepping for the shape rather than the error that fired. The third (Knowledge Base) was found only by asking who consumes the returned count. Fixing the site that threw would have left the corpus-protecting path untouched.

The Architectural Reality

Site 2's window is wider than it looks and does not require another process. Between the counts and the scans the method does await import('fs-extra'), await import('path') and await fs.ensureDir(...). better-sqlite3 blocks writes only on the same connection during iteration — a second connection (another daemon writing one memory) lands in that gap freely.

Reproduced against a real WAL database, one concurrent insert from a second connection:

expected=100  exported=101  (one concurrent insert)
SHIPPED guard  (exported !== expected) -> destroys backup: true
REPAIRED guard (exported <   expected) -> destroys backup: false
negative control — genuine loss (98/101) still aborts under (<): true

Site 3 is annotated in-source as "duplicated deliberately to keep each service's backup logic locally discoverable". That intent is respected below — the predicate is shared, the per-path accuracy caveat stays local.

The Fix

New pure helper ai/services/memory-core/helpers/exportCompleteness.mjs (sibling precedent: vectorWriteInvariant.mjs, vectorJsonlSourceValidation.mjs, graphJsonlImport.mjs in the same folder; the KB service already imports two of them):

• EXPORT_COMPLETENESS — frozen verdicts complete / grew / indeterminate / partial
• classifyExportCompleteness(exported, expected) — pure three-way classification
• recordExportGrowth(stats) — stamps grewDuringExport + growthDelta onto the receipt

indeterminate exists so a non-finite or missing count can never certify a bundle. Defaulting it to complete would let an absent measurement vouch for everything beneath it — the exact shape this module removes. Both callers treat it as fail-loud.

Policy stays at the call sites, because the accuracy caveat differs per path: the vector paths cite offset-paging; the graph path cites Nodes and Edges being separate statements, so the two tables can come from different instants.

Site 3 additionally: counts what it writes, tallies rescue-path skips, and returns exported rather than the snapshot — so verifyBundleIntegrity receives a measurement instead of a restatement.

Contract Ledger Matrix

Decision Record impact

none. Reviewed ADR-0003 / ADR-0017 (Chroma topology), ADR-0015 (graph store backend posture) — this changes neither topology nor backend posture, only completeness accounting inside the export routines. No config leaf is introduced or read, so ADR-0019 does not apply.

Acceptance Criteria

• A grown collection (32272/32271) classifies as grew and the bundle survives
• A genuine shortfall still throws PARTIAL_COLLECTION_EXPORT at all three sites
• A non-finite or missing count classifies indeterminate and fails loud — never complete
• grewDuringExport + growthDelta land on the receipt so a grown bundle never reads as a clean capture
• The KB export returns rows written, tallies rescue-path skips, and no longer returns the snapshot
• Mechanical sunset: a bare exported/expected comparison re-introduced at any export path fails the spec — proven red by injecting the regression (named line 302), then restored
• Pre-existing partial-export specs pass unchanged (graphBackup, backupPath, captureReceipt)
• Post-merge: healthcheck backup.lastSuccessful is non-null on the next scheduled run

Out of Scope

• Making the export atomic. A single-instant snapshot needs a read transaction or a Chroma-side snapshot API; this ticket makes the receipt honest about the window rather than closing it.
• The verifyBundleIntegrity parity rule itself — it now receives a real measurement; whether its threshold is right is a separate question.
• Consolidating the deliberate KB/Memory Core export duplication. That is the concern #16515 measures and #16514 exemplifies; this ticket shares only the predicate.
• Restore-side per-collection reporting — that is #16510.

Avoided Traps

• Fixing only the site that threw. The error named one collection; two more paths carried the same defect, one of them the corpus-protecting one.
• Copying the vector path's rationale to the graph path. Different mechanisms — offset-paging vs two separate statements — so a shared comment would have asserted a false cause.
• Treating growth as clean. Symmetrically wrong; offset-paging can skip one row while duplicating another and still finish high.
• Defaulting an unreadable count to complete. Would let an absent measurement certify the bundle.
• Deleting the duplication. The mirror is deliberate and documented; sharing the predicate keeps that intent while removing the divergence.

Related

• #16510 — restore-side per-collection verdict (same incident, opposite end of the pipeline)
• #16512 — healthcheck blessing an empty knowledge base (@neo-opus-vega)
• #16515 / #16514 — the ai-folder duplication census this deliberately does not pre-empt
• #16428 — the [DATA-SYNC-ALARM] this defect helped keep firing; auto-maintained, not the close target

Live latest-open sweep: checked latest 20 open issues at 2026-08-04T20:53:36Z; no equivalent found. A2A in-flight claim sweep: 30 messages, all read-states — no overlapping [lane-claim].

Origin Session ID: eeacb603-97f1-4241-9b2f-3a542cab6d2c

Retrieval Hint: query_raw_memories("backup export growth partial collection three sites") · PARTIAL_COLLECTION_EXPORT 32272/32271 · branch ada/backup-growth-not-partial