A bundle folder stands in for its export; one failure hides 42 tests

Context

Narrow leaf split out of #16617, which is a multi-AC umbrella about unit specs whose verdict tracks corpus fill rather than the diff. Two of its criteria are about test/playwright/unit/ai/scripts/maintenance/backup.spec.mjs specifically, are independently deliverable in one PR, and are already implemented — this ticket exists so that work has an honest close target instead of riding a draft PR against the umbrella. #16617 stays open for its remaining criteria (the populated-plane/empty-plane negative control, the residual static check, and the check's enforcement reach).

Both defects were measured, not inferred. Both are test-integrity defects: the spec passes while proving less than it claims.

The Problem

Defect 1 — a folder stands in for the export that should fill it. The spec asserted that five bundle subfolders exist:

for (const sub of ['kb', 'mc', 'graph', 'concepts', 'trajectories']) {
    expect(fs.existsSync(path.join(bundleRoot, sub))).toBe(true);
}

runBackup calls fs.ensureDir on every layout folder before any subsystem runs, so existence is guaranteed independently of whether anything landed inside. A subsystem that exported nothing is indistinguishable from one that worked.

The list is also stale: runBackup creates seven folders (backup.mjs:575-588 — kb, mc, graph, concepts, trajectories, mailbox, ledgers), and the loop named five. So mailbox and ledgers sat outside every assertion in the file — and both export nothing in this fixture:

mailbox = {copied: 0, note: "source not present: sent-to-cull.jsonl"}
ledgers = {copied: 0, healAttempts: {copied: 0, note: ...}, healEvents: {...}, recoveryRuns: {...}}

Defect 2 — one failure hides forty-two tests. test.describe.configure({mode: 'serial'}) sat at file scope. Playwright skips the remainder of a serial scope after any failure, so a single fault in the first block aborted every test after it. Measured with an injected beforeAll throw:

1 failed · 42 did not run · 2 passed

A run summary reporting one failure while forty-two tests never executed hides its own blast radius — the reader sees one problem.

The Architectural Reality

• test/playwright/unit/ai/scripts/maintenance/backup.spec.mjs — the subject. Four test.describe blocks: the orchestrator block, capture-lineage (#16404), bundle-meta coherence (#16404), and a corruption-canary block.
• Only the orchestrator block earns the serial constraint. It mutates KB_ChromaManager.getKnowledgeBaseCollection and Memory_StorageRouter.getMemoryCollection / getSummaryCollection across beforeAll/afterAll, so concurrent workers would race on shared module state. The other three dynamic-import pure functions and scope their fixtures to a pid+timestamp temp dir; the nested noticeLegacyBackupRoot block injects its filesystem outright.
• verifyBundleIntegrity (backup.mjs:832) already emits the substrate Defect 1 needs: per-subsystem pass / empty / fail / skipped over RECOVERY_SUBSTRATES = ['kb', 'mc', 'graph'], with empty reserved for zero-zero parity precisely so a zero-row export is not a silent pass. The remaining four subsystems carry {copied: N} copy receipts.
• bundleRoot is read by exactly one test, so the shared fixture carries no cross-test dependence.

The Fix

Defect 1 — assert the property instead of the folder. Each recovery substrate must reach integrity pass with source/bundle parity above zero; each copy subsystem must either have copied rows or carry the note naming the source it lacked. A local copyReceiptIsAccounted helper handles ledgers' nested per-source receipts. The folder check becomes an equality against the bundle's own layout rather than a hand-counted subset.

No row count is pinned for graph. Its size comes from the run-scoped test graph store, so asserting a number would re-introduce exactly the corpus-fill coupling #16617 exists to remove. pass is unreachable at zero, so it carries "exported, and completely" without naming a size.

Defect 2 — move test.describe.configure({mode: 'serial'}) onto the one describe that earns it.

Acceptance Criteria

• Every subsystem in the bundle is asserted to have EXPORTED or to have named the source it lacked; an unexplained copied: 0 fails the spec. Red-proved by mutation: dropping the note from copyJsonlSource's absent-source return must fail with the exact identity.
• The folder assertion covers the bundle's actual layout, not a hand-counted subset — adding an eighth layout folder without asserting it must fail.
• No row count is pinned for graph; the assertion holds whether the test graph store is populated or empty.
• A failure in the orchestrator block no longer aborts the other three blocks. Measured before/after with an injected beforeAll throw, both numbers recorded.
• The serial constraint remains on the block that mutates the KB/MC singleton accessors — the fix must not buy isolation reporting by giving up singleton-race safety.
• The focused spec is green at the delivering head.

Out of Scope

• #16617's remaining criteria — the populated-plane/empty-plane negative control, the residual static check, and its enforcement-reach statement. Those need machine states or a check this ticket does not author.
• copyJsonlSource's unexplained-zero path. backup.mjs:1452 returns {copied: jsonlFiles.length} with no note for a source directory that exists but holds zero .jsonl files (warn-only). That is a genuine production silent-zero, and the new assertion fails on it — but repairing production behavior does not belong in a test-integrity ticket. Recorded on #16617.
• Reaching zero did not run. Playwright skips the remainder of a serial scope after any failure, so the orchestrator block's own tests will still report as skipped when its fixture fails. That residual is a property of a constraint that exists for a real reason; eliminating it means dropping the constraint.

Avoided Traps

• Asserting graph's row count. The observed 17 comes from a run-scoped store. Pinning it would make this spec's verdict track corpus fill — the defect the parent ticket is named after.
• Asserting the observation instead of the property. #16617's body claimed graph exports nothing here; execution falsified that (it exports and reaches pass, because the UNIT_TEST_MODE graphTest formula wires it). Writing the property rather than that observation is what makes the fix survive a dead premise — the mutation that forces the branch the parent ticket described still goes red.
• Converting the orchestrator block's hooks to beforeEach/afterEach to turn skips into failures. Tried and rejected: the did not run count stayed at 22, because the hook type is not what causes the skip. It would have cost nineteen fixture rebuilds for no measured gain.
• Dropping serial entirely. It would zero the skip count and reintroduce a singleton race across concurrent workers.

Related

#16617 (parent umbrella; this leaf splits two of its criteria) · #16404 (the capture-lineage and bundle-meta blocks this must not disturb) · #10129 (the orchestrator block's origin) · #16417 (staging/publish criteria in the same block)

Decision Record impact: none — test-only; no ADR authority touched.

Agent OS Structure Map gate: N/A — no .mjs file is created or relocated. The spec already lives at the canonical unit path for Brain-side tests (test/playwright/unit/ai/scripts/maintenance/), per unit-test workflow §7 and its siblings in that folder.

Live latest-open sweep: checked the latest 20 open issues plus a scoped title grep across 100 open issues on backup|serial|did not run|subfolder|spec at 2026-08-24T15:37Z — no equivalent found. A2A in-flight claim sweep over the last 30 messages (all read-states): no competing [lane-claim]/[lane-intent] on this scope.

Origin Session ID: cad88c79-073f-4816-aaa7-e779224f2af3

Retrieval Hint: query_raw_memories("backup.spec.mjs folder existence vacuous serial file scope 42 did not run") · falsification anchors: verifyBundleIntegrity in ai/scripts/maintenance/backup.mjs, and test.describe.configure placement in the spec.