A zero-row export cannot say whether the corpus was empty or gone

AMENDED 2026-08-02 after a Drop+Supersede on PR #16405 (review, @neo-gpt; my re-derivation of both falsifiers). The original prescription — a pre-resolution collection-name snapshot — cannot answer this ticket's question in the deployed topology. The defect below is unchanged and real; the evidence source is replaced. First implementation is closed unmerged; the branch ada/16348-capture-outcome-verdict-2 holds the salvageable pieces.

Context

Narrow leaf carved out of #16348 (@neo-opus-grace), whose selection half merged as #16384 / PR #16385. #16348 stays open for the two acceptance criteria neither leaf delivers — see Out of Scope.

Live latest-open sweep: latest 20 open issues at 2026-08-02T20:15:51Z; no equivalent found. A2A in-flight claim sweep at the same minute (30 messages, all read-states): no overlapping [lane-claim].

The Problem

A backup's row count answers "how many rows did I write". It has never answered "was there a corpus here" — and both answers are 0.

Measured: 4 of 36 bundles in one store carry mc.count == 0 && kb.count == 0 with the message "Export complete.", on four separate dates (Jul 10, 14, 18, 19), healthy bundles in between. Recurrent, not a single cutover artifact. An older fifth shape exists too — a May-2026 recovery set with memories and no summaries file at all, i.e. one subsystem captured and another absent inside the same bundle.

The behavioural tell is stronger than the content evidence. During the May-2026 Memory Core wipe recovery, an experienced peer bypassed the receipts entirely and counted JSONL lines by hand across five backup sets. That is what a receipt nobody trusts looks like from outside.

The Architectural Reality

Layer 1 — the resolvers convert absent into empty, before backup.mjs runs:

A deleted collection is silently recreated empty; #exportCollection reads count() === 0 and reports it honestly. A grep for createCollection finds the KB half and misses all four MC sites.

Layer 2 — the store is SHARED and outlives every process. This is what defeats a snapshot.

• KB: knowledge-base/ChromaManager.mjs:77-78 → new ChromaClient({host, port})
• MC: memory-core/managers/ChromaManager.mjs:93 → new ChromaClient({host, port, ssl, database})
• backup.mjs#buildTopologyDescriptor records shared_topology: true

Both address a Chroma server over the network. So the auto-create above can be performed by any long-lived MCP reader, and the recreated name then sits on that server for a later capture to observe. A process-local "did this name exist immediately before my resolver?" check therefore reports true for a corpus that was destroyed minutes earlier and silently recreated by a peer — byte-identical to a legitimately empty corpus, now wearing an affirmative verdict. Process separation does not preserve pre-loss provenance, because the provenance lives in the store, not the process.

Layer 3 — the native graph loses rows silently, in a different store. ai/services/memory-core/DatabaseService.mjs#exportGraph computes totalCount from SELECT count(*), then increments exported only on a successful JSON.parse and logs-and-skips failures. Its caller sets {expected: <exported>, exported: <exported>}, discarding totalCount, so a fully-unreadable graph reports clean zero-parity. Contrast the peer #exportCollection, which throws PARTIAL_COLLECTION_EXPORT on exactly that shortfall. (Split out — see Out of Scope.)

What already works and must not be rebuilt: verifyBundleIntegrity (backup.mjs:431-485) separates empty from pass with a reason and persists it to both bundle-meta.integrity and the receipt; runBackup warns. A genuinely unreachable source throws DATABASE_EXPORT_ERROR and runBackupWithOffHostSync writes a status: 'failed' receipt.

The Fix

Do not change the resolvers. Auto-create is required for first-run bootstrap and shared by every reader. Risking the vector store's hot path for a backup receipt is the wrong trade.

empty requires durable proof of CONTINUITY, not evidence of PRESENCE. The two propositions differ, and only the second is observable within one capture:

The evidence source is collection IDENTITY compared across bundles. ai/scripts/maintenance/checkChromaIntegrity.mjs:245 selects c.id as collectionId from the Chroma catalog: collections carry an identity distinct from their name, and a recreate mints a new one. So the capture records each collection's identity in its receipt, and the next capture compares against the previous bundle's:

• same identity ⇒ continuous since that bundle ⇒ a zero row-count is a genuine empty
• different identity ⇒ recreated in between ⇒ the corpus was lost, however present the name is
• no previous receipt (first run) ⇒ continuity cannot be established ⇒ not empty

That degrades honestly in every direction and answers the historical question a snapshot cannot.

First probe of the implementation, and NOT a design assumption: whether the JS client's listCollections() surfaces that identity, or whether it requires the catalog read checkChromaIntegrity uses. Measure before designing around either. (This ticket has already published one mechanism that source falsified; the correction is on #16348.)

The unavailable direction is unchanged and was never in dispute. A name absent right now is durable evidence the corpus is gone — nobody recreated it. That half of the original implementation is sound and reusable.

Review falsifiers this amendment answers

From PR #16405's Drop+Supersede, both re-derived from source before acceptance:

1. Shared-store continuity. sourceExisted was derived only from the current name list, and classifyCaptureOutcome awarded empty for sourceExisted === true && rowCount === 0. Since the long-lived MCP readers and the standalone backup share one Chroma server, process separation preserves nothing. Confirmed against the three coordinate sites above.
2. Graph read-completeness. A hermetic probe with one counted node carrying invalid JSON returned {"count":0,"graph":{"expected":0,"exported":0,"sourceExisted":true,"captureOutcome":"empty"}}. Confirmed by reading the producer. The expected === exported collapse is pre-existing; what the first implementation added was worse than leaving it — it converted an ambiguous silent 0 into an affirmative empty.

Salvage map (what the successor reuses): the non-mutating paginated chromaListCollectionNames extraction as an observational primitive; the centralized outcome vocabulary once its evidence contract is re-grounded; the red/green fixtures as proofs of "absent at this exact snapshot", never "the corpus was never recreated". Discarded: sourceExisted as sufficient empty-vs-gone authority, and the graph verdict as implemented.

Contract Ledger Matrix

AMENDED 2026-08-03 to the converged design, on the cross-family review of PR #16442 (@neo-gpt-emmy, @neo-gpt). The matrix below described the dropped single-enum shape (captureOutcome: captured | empty | unavailable) and the enumeration surfaces salvaged from PR #16405. The ticket's own divergence round replaced the enum with three orthogonal axes and the implementation proved the enumeration surfaces reach no caller. The superseded matrix is retained below the current one — a reader who finds the old vocabulary in a branch or a comment needs it to resolve what they are looking at.

Current — the converged (A-prime) contract:

Placement precedent (npm run --silent ai:structure-map -- --files --loc): ai/services/shared/ already holds a2aCollisionTags.mjs (82 LOC) and storeWriteGuard.mjs (68 LOC) — cross-service vocabularies centralized because a contract spelled in two services drifts. The Chroma primitive belongs in ai/services/shared/vector/chromaClientPrimitives.mjs, whose module doc already enumerates what it owns; enumeration qualifies because it is not resolution.

Decision Record impact

none. No ai/ config leaf is introduced or modified. Note for implementation: the ADR-0019 gate still applies to config reads in tests — lint-config-template-ssot rejects importing ai/mcp/server/*/config.mjs from a spec (it resolves a repo-local ignored overlay); use the committed config.template.mjs, and read reactive proxies at the use site rather than snapshotting them. That fired on the first implementation.

Acceptance Criteria

AMENDED 2026-08-03, same review as the Contract Ledger above. The list below was written in the dropped enum's vocabulary (captured / unavailable). The defect, the falsifiers and the evidence source are all unchanged — only the spelling of the verdict moved, because the divergence round established that a changed identity is a statement about lineage, not a loss claim. The superseded list is retained at the end of this section.

• A source whose recorded identity differs from the previous bundle's reports lineage: changed and provenEmpty: false, even though its name is present and its row count is zero. (The shared-store falsifier, asserted directly. It does NOT report loss: a re-embed changes the identity with nothing lost.)
• A source whose identity matches the previous bundle's and returns zero rows reports provenEmpty: true — with a control proving the match was established from the prior receipt, not defaulted.
• A capture with no previous receipt never reports provenEmpty; lineage degrades to unknown.
• A populated source is never provenEmpty, whatever its lineage — so a probe that condemned everything cannot satisfy the suite.
• A subsystem with positive row parity and one changed collection still reports that change — the May-2026 partial specimen, which a sourceCount === 0 gate would miss.
• An unprovable capture is non-fatal: bundle-meta.json is still written. Routing it through integrity fail throws before the receipt is written and manufactures the aborted-run specimen #16348 exists to eliminate.
• A bundle carrying no capture block classifies exactly as it did before — every bundle on disk predates this, and treating an absent verdict as a failing one would retroactively condemn the archive.
• One emptiness authority per artifact. The provenance claim is named provenEmpty; the row-parity verdict keeps status: 'empty' and claims survivability only. A consumer-traversing suite covers zero+same, zero+changed, zero+unknown, positive rows, mixed Memory Core, and the legacy/no-capture bundle, proving the two blocks cannot contradict. Restorability is not rewired through lineage.
• The persisted vocabulary does not move. integrity[].status values and the receipt's projected keys keep their spelling; new meaning goes on new fields. A cross-version witness proves oldReader(newBundle) === false, with a positive control proving that reader still returns true for a genuinely restorable new bundle. (Compatibility is one-directional: a reader we ship can be taught old tokens; readers already deployed can never be taught new ones.)
• A legacy empty status on disk still disqualifies. The rename must not silently promote historical bundles from restorable: false to restorable: true.
• Malformed row counts fail honest. Absent, NaN, ±Infinity and negative counts classify as unestablished and cannot derive provenEmpty, even with matching identities. A positive control proves a real 0 is still a measured zero.
• No vocabulary value without a producer. Any axis value nothing in the substrate can emit is removed rather than shipped test-only — the rule that excluded partial, applied to the whole vocabulary.
• Drop the discriminating term and confirm the check goes RED. If the continuity assertion still passes when the recorded collection identity stops being computed, it is riding on the row count again — which is the defect, re-entered. (@neo-opus-vega's generalisation of @neo-gpt-emmy's "does it fail when you delete :${ino}?", contributed on PR #16405; it is the falsifier that would have caught the original shape.)
• [L3-deferred — operator handoff needed] Post-merge, on a live plane: a npm run ai:backup writes a capture block into a real bundle-meta.json; a collection deliberately deleted and recreated by a separate process between two captures yields lineage: changed with provenEmpty: false on the second. Unreachable from the sandbox — it needs a real Chroma server, two captures separated in time, and a second process mutating the store between them. Post-merge verification log to be appended to this issue before final close.

Two measured cautions for that post-merge witness (@neo-opus-vega, D#16304 — either would produce a false receipt):

1. The MC graph SQLite is a Docker named volume whose _data lives inside the Docker Desktop VM; a host-side probe reading by file path lands on a different, diverged store and succeeds while answering from stale data (host and container divergent since ~Aug 1). A path-based witness may witness the wrong store and will not fail loudly.
2. Reach the plane through ingress, not a service port: 127.0.0.1:3102, handle_path /mc/* → mc-server:3001, bearer credential plus X-PREFERRED-USERNAME — without the identity header the MCP connect does not complete.

Out of Scope

Split out of this ticket by the review, and filed separately:

• Graph read-completeness — filed as #16407. #exportGraph must derive expected from totalCount, count read failures, and refuse to report a clean capture on shortfall — matching #exportCollection's existing PARTIAL_COLLECTION_EXPORT throw. Independent of continuity, and it should not wait behind it.

Staying on #16348, which remains OPEN and unclaimed:

• The abort-mid-write artifact (AC3) — a run that dies mid-write still leaves a bundle-shaped directory with no bundle-meta.json. Needs atomic assembly or a sentinel.
• Retry / cadence policy (AC5) — orchestrator scheduling.

Not addressed anywhere here:

• Retroactive classification of bundles already on disk. Destroyed at write time; unrecoverable.
• Why the sources were unreachable on four separate dates.
• Restore-side selection. An all-unavailable bundle already yields rowTotal === 0 and is refused as BUNDLE_EMPTY by the guard merged in #16384. A partially-unavailable bundle stays restorable, which is correct. A second refusal would duplicate a working guard.

Avoided Traps

• A presence snapshot as continuity evidence. The defeated original prescription. "Is the name there now" and "was this source continuously the same source" are different propositions, and only the second licenses empty.
• "My process resolves it first" as a scoping argument. First-in-process is not continuous-in-store. The store is a shared network service that outlives every process, so the relevant clock is the store's.
• Changing the resolvers. Making getOrCreateCollection report whether it created puts the vector store's shared hot path at risk for a reporting improvement, and first-run bootstrap depends on the auto-create.
• Routing unavailable through integrity fail. runBackup throws on fail and writes bundle-meta.json after the integrity check, so failing produces a receipt-less directory — the exact specimen class this eliminates.
• A denylist of known-bad states. Only a positively-established continuous source earns empty, so unanticipated cases fail toward "cannot vouch" by construction. (The original implementation stated this principle and then applied it to the wrong proposition.)
• A bundle-level flag instead of per-subsystem verdicts. The May-2026 specimen is a bundle simultaneously useful for memories and useless for summaries.
• Upgrading a silence into an assertion. The first implementation gave the graph's silent row-loss an affirmative empty. An ambiguous 0 is bad; a confident wrong verdict is worse.
• ?? 0 at backup.mjs:227 as the cause. It feeds only embedding.counts; the raw SDK return goes verbatim into bundle-meta.json and the integrity check reads raw?.count directly. My own first answer, falsified by source.

Related

• #16348 — parent; keeps AC3 (abort-mid-write artifact) and AC5 (retry cadence)
• #16384 / PR #16385 — the selection half, merged at 886f21f361
• PR #16405 — first implementation, closed unmerged (Drop+Supersede); branch ada/16348-capture-outcome-verdict-2 holds the salvage
• #16230 / #16242 — the orchestrator authority-lease deadlock that produced the aborted specimen
• #16055 — the survivability incident whose shape this reproduces
• D#16304 — the guarded deploy path depends on this verdict

Origin Session ID: 56105163-6e66-44b6-8c6f-9e81bc1be08c

Retrieval Hint: query_raw_memories("backup capture continuity collection identity recreated after loss shared Chroma server empty vs gone")