Context
Scoped deliberately out of #16201 (which relocates the backup root out of the working tree) so that ticket stayed one change. Filed separately because it is a distinct failure mode with a distinct remedy.
Observed on the canonical machine 2026-07-30: 36 bundles, ~133 GB, spanning 2026-07-01 → 2026-07-30, sitting on the same filesystem as memory-core-graph.sqlite (728 MB, plus a 42 MB WAL) — the graph backing roughly 30,000 memories.
Live latest-open sweep at 2026-07-30T23:05Z: searched open issues for backup/disk/graph same-failure-domain topics; zero matches. A2A claim sweep at the same time: no competing claim. Sweep caveat: ChromaDB is down (see #16167), so semantic retrieval was unavailable; this sweep is live-GitHub-and-source only and claims no semantic-memory miss.
The Problem
The corpus that exists to survive a graph loss is stored where it can cause one.
- ~133 GB of bundles grow daily against a graph of 728 MB. The protector is ~180× the size of the protected.
- If the disk fills, SQLite writes fail. The graph stops accepting memories and the next bundle cannot be written — both halves of the durability story fail from the same cause, at the same moment.
- There is no off-host copy:
offHostSync.status is "disabled" in the live receipt, and #16199 shows a required-and-unmet posture still exits 0 (PR #16200 approved, awaiting merge).
The compose profile already names this principle — for the cloud case only:
ai/deploy/docker-compose.yml:272 — "container recreate. This named volume is durability, not off-host backup."
and configBase.mjs:1299-1300 justifies the cloud requirement as "the named volumes and the host that carries them are the same failure domain." That reasoning is equally true of a maintainer machine, where the local profile defaults the requirement off because "the operator's own machine is not a durability boundary we can reason about" — which is an argument for not co-locating, not for accepting it silently.
Separation of observation from inference: the sizes, dates, bundle count and receipt status are directly observed. The disk-fill consequence is inference from SQLite write behaviour under ENOSPC; no fill event has occurred, so this is a latent risk rather than an incident.
The Architectural Reality
ai/configBase.mjs:198 — backupPath: leaf(path.resolve(planeDataRootDefault, 'backups'), 'NEO_BACKUP_PATH', 'string', {planeMember: true}), and :15 resolves planeDataRootDefault from neoRootDir. So bundles and graph share a root by construction, not by operator choice. #16201 moves that root out of the working tree; it does not separate it from the graph.
maintenance.backup.retention is {keepMinimum: 3, maxDays: 30}, and the observed spread is exactly 30 days — so retention appears to be working. The 133 GB is the intended steady state, not a pruning bug. That is what makes this structural rather than a leak to fix.
- Nine of ten clones symlink
.neo-ai-data members to the canonical checkout, so this is one shared disk for the whole swarm's history, not a per-seat concern.
ai/scripts/maintenance/backup.mjs owns bundle creation and retention; restore.mjs:730's verifyLatestBackupRestorable is the read-only restorability probe (never yet run against these bundles).
The Fix
Direction rather than prescription, because the right answer depends on OQ1:
- Fail before filling. A pre-flight free-space check in the backup lane that refuses to start a bundle it cannot complete, and reports the refusal — rather than a partial write competing with the graph for the last blocks.
- Surface headroom as an observable, alongside the existing durability posture, so "the disk is 90% full" is a reported state rather than something discovered when writes fail.
- Separate the failure domains — the actual remedy, and the subject of OQ1.
Open Questions
- OQ1 — separate volume, smaller retention, or off-host as the real answer? Three candidate shapes: (a) bundles on a distinct volume/disk from the graph; (b) reduce
maxDays/keepMinimum so the corpus stays small relative to free space; (c) treat off-host sync (#16199) as the durability answer and keep only a short local window. These are not exclusive and they have different costs. Falsifier for (b): a shorter window narrows the recovery horizon, which is the thing the corpus exists to widen — so it trades one risk for another rather than removing one.
- OQ2 — what is the actual free-space margin today? I observed sizes but not filesystem capacity or headroom. Without that, "133 GB is dangerous" is a ratio argument, not a measurement. This should be answered before choosing among OQ1's options.
- OQ3 — does bundle size scale with graph size or with churn? A 728 MB graph producing ~3.6 GB bundles suggests bundles carry more than the graph (Chroma vectors, WAL, ledgers). If growth tracks churn rather than graph size, the trajectory matters more than the current figure.
- OQ4 — should this gate the #16167 cutover? Probably not: the cut replaces the container data plane and does not change disk topology. Naming it so the answer is deliberate rather than assumed.
Acceptance Criteria
Out of Scope
- Relocating the backup root out of the working tree — #16201 owns that. It removes a deletion vector; this ticket addresses a capacity vector. Neither subsumes the other.
- Off-host sync enforcement (#16199) and first-boot restore (#15639).
- Bundle format, compression, or what a bundle contains.
- Chroma store bloat (#14079) and the unused FTS5 index (#14192) — both would reduce bundle size as a side effect, and both are separately owned.
Avoided Traps
- Treating this as a retention bug. Retention is working — 36 bundles across exactly 30 days against
maxDays: 30. The 133 GB is the designed steady state, which is precisely why the co-location is structural.
- Shrinking the window as a reflex. It reduces disk pressure by reducing the recovery horizon, i.e. by making the thing worse at its job. Acceptable only as a deliberate trade with the horizon stated.
- Assuming a full disk only loses backups. SQLite needs space to write;
ENOSPC takes the graph and the next bundle together. That simultaneity is the whole finding.
Related
- #16201 — backup root inside the working tree (the deletion vector; same corpus, different failure mode)
- #16199 / PR #16200 — required off-host durability now fails loudly
- #15639 — first-boot restore from the latest bundle
- #14079 — Memory Core Chroma store 2.5 GB bloat
- #14192 — unused Chroma FTS5 index (~495 MB)
- #16167 — the cutover this deliberately does not gate (OQ4)
Origin Session ID: 0a7f5f1d-cf12-4698-984c-17b64eea5178
Retrieval Hint: backups 133GB same disk as memory-core-graph.sqlite ENOSPC same failure domain free-space preflight retention 30 days working as designed
Context
Scoped deliberately out of #16201 (which relocates the backup root out of the working tree) so that ticket stayed one change. Filed separately because it is a distinct failure mode with a distinct remedy.
Observed on the canonical machine 2026-07-30: 36 bundles, ~133 GB, spanning 2026-07-01 → 2026-07-30, sitting on the same filesystem as
memory-core-graph.sqlite(728 MB, plus a 42 MB WAL) — the graph backing roughly 30,000 memories.Live latest-open sweep at 2026-07-30T23:05Z: searched open issues for backup/disk/graph same-failure-domain topics; zero matches. A2A claim sweep at the same time: no competing claim. Sweep caveat: ChromaDB is down (see #16167), so semantic retrieval was unavailable; this sweep is live-GitHub-and-source only and claims no semantic-memory miss.
The Problem
The corpus that exists to survive a graph loss is stored where it can cause one.
offHostSync.statusis"disabled"in the live receipt, and #16199 shows a required-and-unmet posture still exits 0 (PR #16200 approved, awaiting merge).The compose profile already names this principle — for the cloud case only:
and
configBase.mjs:1299-1300justifies the cloud requirement as "the named volumes and the host that carries them are the same failure domain." That reasoning is equally true of a maintainer machine, where the local profile defaults the requirement off because "the operator's own machine is not a durability boundary we can reason about" — which is an argument for not co-locating, not for accepting it silently.Separation of observation from inference: the sizes, dates, bundle count and receipt status are directly observed. The disk-fill consequence is inference from SQLite write behaviour under
ENOSPC; no fill event has occurred, so this is a latent risk rather than an incident.The Architectural Reality
ai/configBase.mjs:198—backupPath: leaf(path.resolve(planeDataRootDefault, 'backups'), 'NEO_BACKUP_PATH', 'string', {planeMember: true}), and:15resolvesplaneDataRootDefaultfromneoRootDir. So bundles and graph share a root by construction, not by operator choice. #16201 moves that root out of the working tree; it does not separate it from the graph.maintenance.backup.retentionis{keepMinimum: 3, maxDays: 30}, and the observed spread is exactly 30 days — so retention appears to be working. The 133 GB is the intended steady state, not a pruning bug. That is what makes this structural rather than a leak to fix..neo-ai-datamembers to the canonical checkout, so this is one shared disk for the whole swarm's history, not a per-seat concern.ai/scripts/maintenance/backup.mjsowns bundle creation and retention;restore.mjs:730'sverifyLatestBackupRestorableis the read-only restorability probe (never yet run against these bundles).The Fix
Direction rather than prescription, because the right answer depends on OQ1:
Open Questions
maxDays/keepMinimumso the corpus stays small relative to free space; (c) treat off-host sync (#16199) as the durability answer and keep only a short local window. These are not exclusive and they have different costs. Falsifier for (b): a shorter window narrows the recovery horizon, which is the thing the corpus exists to widen — so it trades one risk for another rather than removing one.Acceptance Criteria
Out of Scope
Avoided Traps
maxDays: 30. The 133 GB is the designed steady state, which is precisely why the co-location is structural.ENOSPCtakes the graph and the next bundle together. That simultaneity is the whole finding.Related
Origin Session ID: 0a7f5f1d-cf12-4698-984c-17b64eea5178
Retrieval Hint:
backups 133GB same disk as memory-core-graph.sqlite ENOSPC same failure domain free-space preflight retention 30 days working as designed