Publish backup bundles only after capture completes

Context

#16348 AC3 remains after its newest-valid selection half merged and its capture-continuity half moved to #16404. Live source inspection on current origin/dev found a deterministic path from an ordinary export failure to a restore-authorized torn bundle; this is not limited to an abrupt process kill.

Live latest-open sweep: checked the latest 20 open issues at 2026-08-02T22:17Z; no equivalent ticket exists. The closest artifacts are #16348 (the parent problem ledger) and #16404 (collection continuity, a different mechanism). A2A all-read-state sweep over the latest 30 messages found no overlapping lane claim; Ada’s active plan round is explicitly scoped to #16404.

The historical contract already calls this an atomic timestamped bundle: #10129 established one backup-<timestamp>/ as the indivisible recovery unit. This ticket completes that contract at the publication boundary rather than changing the bundle format.

The Problem

A final restore-candidate directory becomes visible before the capture has earned that status.

ai/scripts/maintenance/backup.mjs currently:

1. resolves the final backup-* path (runBackup, around line 263);
2. creates its complete directory tree before the first export (around line 288);
3. streams KB, MC, graph, and file-backed members directly into that final tree;
4. runs integrity verification only after those writes (around line 342);
5. writes bundle-meta.json only after every export and integrity check succeeds (around line 378).

If an export throws after earlier members were written, runBackupWithOffHostSync() writes last-backup-receipt.json at the backup root with bundleName: null; the torn directory itself carries no failure identity.

The restore side compounds that publication error. validateBundle() treats an absent bundle-meta.json as a legacy bundle, returns a synthetic {legacy: true} receipt, and probeBundle() authorizes RESTORABLE whenever the rows it streamed total above zero. A partial capture can therefore shadow a complete older recovery source while accurately being the newest backup-* directory.

The Architectural Reality

• Publication belongs to the producer. Teaching every consumer another torn-bundle heuristic leaves the final namespace lying. runBackup() owns when a candidate becomes a candidate.
• The final directory name is the recovery admission surface. Restore, retention, health, deployment preflight, and off-host sync all discover or identify bundles through the published backup-* path.
• bundle-meta.json is completion evidence, but today it is not the publication fence. It lands after integrity verification inside a directory that has been externally visible throughout the run.
• The same-filesystem parent provides the atomic primitive. A staging directory created beside the destination can be renamed into the final name only after the meta receipt exists. No cross-store transaction is claimed.
• Owning placement: existing ai/scripts/maintenance/backup.mjs (structure map: ai/scripts/maintenance/, existing file; no new .mjs placement). Existing unit witnesses live in test/playwright/unit/ai/scripts/maintenance/backup.spec.mjs and restore.spec.mjs.

The Fix

Make runBackup() a publish-on-complete transaction over directory identity:

1. Resolve the requested final root exactly as today and fail loud if it already exists; never overwrite a prior recovery source.
2. Create a unique sibling staging directory whose basename cannot match the root-level backup-* candidate filter, while retaining the intended final basename for diagnosis (for example .backup-partial-<final-name>-<unique>).
3. Point the entire current layout and every export at the staging root.
4. Write the existing bundle-meta.json in staging after the existing integrity gate and await write completion. This is the publication fence; it does not claim power-loss durability because the current writer does not fsync the files or directory.
5. Atomically rename the same-filesystem staging directory to the final root.
6. Return only the final bundleRoot; runBackupWithOffHostSync() therefore cannot sync or receipt-name a bundle before publication.
7. Remove the owned staging directory on caught failures and rethrow. An uncatchable process termination may leave a .backup-partial-* directory, but it is self-identifying and outside every backup-* recovery candidate set by construction.

The final bundle layout and bundle-meta.json schema do not change.

Contract Ledger Matrix

Surface anchors were verified against current source before filing; the proposed staging namespace is new and owned by this ticket.

Decision Record impact

none. This completes the existing atomic-bundle publication contract and adds no config, service boundary, MCP surface, or persistent schema field.

Acceptance Criteria

• Before all exports, integrity verification, and bundle-meta.json complete, the final bundleRoot does not exist.
• A deterministic mid-capture export failure leaves no final backup-* directory and rethrows the original failure; the wrapper still writes its existing root-level failed receipt and never invokes off-host sync.
• A success path atomically publishes the final root, returns that exact path, preserves the current subdirectory/meta layout, and leaves no staging directory.
• An existing final destination fails loud without overwriting or mutating it.
• An abrupt-death-shaped .backup-partial-* fixture containing parseable recovery rows but no meta is never examined by verifyLatestBackupRestorable(); an older valid published bundle remains the selected recovery source.
• Explicit bundleRoot overrides retain their current caller-visible semantics while staging beside that destination on the same filesystem.
• The discriminating publication term is mutation-checked: writing directly to the final root (or changing the staging basename into the backup-* candidate namespace) makes the focused guard fail.
• [L3-deferred — live scheduled backup needed] Post-merge: the next scheduled backup exposes no final directory until completion, then reports the same final path in its success receipt and off-host sync input.

Out of Scope

• #16348 AC5 retry/cadence policy. It needs an explicit retry clock, budget, restart persistence, config authority, and exhausted-budget observability before implementation.
• #16404 collection-identity continuity and captured / empty / unavailable semantics.
• Reclassifying or deleting historical torn backup-* directories already on disk.
• Automatic retention of abrupt-death .backup-partial-* residue. It is excluded from recovery authority here; a cleanup policy should be evidence-led rather than silently deleting forensic artifacts.
• Changing the final bundle layout, receipt schema, restore import semantics, or off-host transport.

Avoided Traps

• Restore-only heuristic. Rejecting one observed torn shape leaves future consumers and new partial shapes exposed; publication must be truthful at the owner.
• Writing a failure receipt after the fact. A hard process termination cannot run a catch block. The staging name must be non-authoritative before the first byte is written.
• Deleting the final directory on error. It is briefly authoritative and creates a race in which a consumer can observe it. Never publish is stronger than publish-then-delete.
• Cross-filesystem move/copy fallback. Atomicity relies on staging beside the destination and using rename semantics; a copy fallback recreates the partial-visibility defect.
• Folding retry policy into this repair. Artifact publication and scheduler cadence have different authorities and falsifiers.

Related

• Parent problem ledger: #16348
• Capture-continuity sibling: #16404
• Newest-valid selection shipped by PR #16385
• Native graph export completeness shipped by PR #16409
• Original atomic-bundle contract: #10129

Origin Session ID: 97816bde-fe7c-4b23-98c4-fbe4d89ed53d

Retrieval Hint: query_raw_memories("backup torn directory bundle-meta absent atomic publish staging restore legacy RESTORABLE")