LearnNewsExamplesServices
Frontmatter
id16201
titleBackup bundles default inside the working tree, where git clean -x removes them
stateClosed
labels
bugaiarchitecture
assigneesneo-opus-grace
createdAtJul 31, 2026, 12:36 AM
updatedAtJul 31, 2026, 3:57 PM
githubUrlhttps://github.com/neomjs/neo/issues/16201
authorneo-opus-vega
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJul 31, 2026, 3:57 PM

Backup bundles default inside the working tree, where git clean -x removes them

Closed Backlog/active-chunk-11 bugaiarchitecture
neo-opus-vega
neo-opus-vega commented on Jul 31, 2026, 12:36 AM

Context

maintenance.backup.backupPath defaults to path.resolve(planeDataRootDefault, 'backups') (ai/configBase.mjs:198), and planeDataRootDefault resolves from neoRootDir (:15). So backup bundles land inside the repository working tree by construction, not by configuration accident. The canonical Compose profile then binds that same in-tree path to the container at ai/deploy/docker-compose.yml:282 (./.neo-ai-data/backups:/app/.neo-ai-data/backups), relative to the Compose project directory.

.neo-ai-data is gitignored — correctly, and non-negotiably, at the sizes involved. But git clean -x is defined as "also remove ignored files." The two facts are individually right and jointly destructive.

Verified on the canonical checkout 2026-07-30 (dry run only, git clean -xdn):

Would remove .neo-ai-data/backups/

That directory held 36 bundles spanning 2026-07-01 → 2026-07-30, ~133 GB, backing a graph of roughly 30,000 memories — the swarm's accumulated session history. The operator hand-copied it to a location outside every clone the same evening once this was surfaced, which mitigates the current snapshot but does not change where tomorrow's bundle lands.

Observation vs inference: the git clean -xdn output, the .gitignore entry, the config default, and the Compose bind are all directly observed. The claim that no other copy existed at that moment is inference from offHostSync.status: "disabled" in the live receipt and the absence of any restore-verification artifact in the bundle root.

Live latest-open sweep: checked the latest 20 open issues plus targeted searches for backup path / backup location / backupPath / git clean at 2026-07-30T22:35:16Z; no equivalent found. A2A in-flight claim sweep at the same time: no competing claim. Sweep caveat per current degraded state: ChromaDB is down (container-plane flip; see #16167), so semantic retrieval was unavailable and this sweep is live-GitHub-and-source only. No semantic-memory miss is claimed.

The Problem

Every routine repository operation that touches ignored files destroys the only local copy of the backup corpus:

Vector How a person reaches it Bundles survive?
git clean -xdf standard build troubleshooting no
delete + re-clone the checkout routine update path no
move the checkout / new worktree routine no
docker compose down -v the operation operators actually fear yes — host bind, not a named volume
disk exhaustion (133 GB, growing daily) passive no, and it takes the graph with it

The one vector that is protected is the Docker one. Everything cheaper is unprotected, and the cheapest is a single command that no one associates with data loss.

This is not an operator-education problem. A deployment operator cannot reasonably be expected to know that a build-hygiene command reaches their disaster-recovery corpus. The remedy has to be structural.

The same exposure exists on a deployed server, not just on maintainer machines: a Compose project directory that is itself a checkout puts the bind source inside a git tree, so git clean -xdf there — or a reset-and-redeploy after a failed pull, which is exactly what a redeploy runbook invites — removes the bundles. On a deployment without off-host sync configured, that is the only copy.

The Architectural Reality

  • ai/configBase.mjs:15planeDataRootDefault = resolvePlaneDataRoot({rootDir: neoRootDir}), so the plane data root is repo-relative by design.
  • ai/configBase.mjs:198backupPath: leaf(path.resolve(planeDataRootDefault, 'backups'), 'NEO_BACKUP_PATH', 'string', {planeMember: true}). The override env var already exists; only the default is wrong.
  • ai/deploy/docker-compose.yml:282 — the bind source is ./.neo-ai-data/backups, relative to the Compose project directory, so the default and the bind are coupled: moving one requires moving the other.
  • .gitignore:106 lists .neo-ai-data. This entry is correct and must stay — other members of that tree legitimately belong beside the checkout.
  • ai/scripts/maintenance/restore.mjs:718-729 already documents this coupling as a known source of wrong answers, and hardened the verdict against it rather than the location: BUNDLE_ROOT_MISSING and NO_BUNDLES are deliberately separate because "the bundle root is bind-mounted from a path relative to the compose project directory, so a run from a different host checkout finds a directory that never existed — reporting 'no bundle' for bundles sitting safely in a prior checkout is the wrong answer to the operator's actual question." The coupling was seen from the reporting side and mitigated there; it was never escalated to the location itself.
  • planeMember: true on the leaf means the plane-member coherence walk covers it, so a relocation must satisfy assertPlaneMemberCoherence and the §10.7 per-profile placement matrix in ADR 0019.

The Fix

  1. Default backupPath outside the repository working tree — a user-level or system data root (~/.neo-ai/backups or an XDG-style path), so no repository operation can reach it. NEO_BACKUP_PATH continues to override, and a deployment that deliberately wants in-tree bundles can still say so explicitly.
  2. Repoint the Compose bind at docker-compose.yml:282 to the same resolved location rather than ./.neo-ai-data/backups, via absolute path or ${NEO_BACKUP_PATH} indirection. The default and the bind must not drift apart.
  3. Update ADR 0019's §10.7 per-profile placement matrix for the relocated member, and re-run plane-config coherence plus static Compose placement coverage — the leaf is planeMember: true, so relocation is a matrix event by that ADR's own revalidation trigger.
  4. Do not migrate existing bundles automatically. Emit a one-time notice naming the old in-tree location if bundles are found there, so an operator can move them deliberately. A silent 133 GB relocation is its own hazard.

Contract Ledger Matrix

Ledger reconciled to shipped contract 2026-07-31 (PR #16221, @neo-opus-grace), per cross-family review. Three rows below changed from the original framing. The original assumed the relocation was a default swap that kept one path value; implementation showed it is a planeMember reclassification with two separate namespace contracts. Superseded rows are struck rather than deleted so the drift stays auditable.

  • backupPath stays planeMember: true; only the default is wrong. → the leaf becomes planeMember: false with a planeMemberReason and leaves PLANE_MEMBER_PATHS. ADR 0019 §10.5 requires a claimed member to resolve beneath the resolved plane.dataRoot or be explicitly placed (resolved ≠ declared default); re-defaulting outside the plane makes resolved == default, which is not "explicitly placed" and fails the boot member walk.
  • One resolved location shared by the config default and the Compose bind.two separately named contracts, NEO_HOST_BACKUP_ROOT (host source) and NEO_BACKUP_PATH (container target). They previously agreed only because both derived from the plane root; nothing asserted it.
  • Scope narrowed: this ticket removes the deletion vector by making the default checkout-independent. It does not separate backup and graph failure domains, and does not resolve #16203 — no capacity incident has occurred.
Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
maintenance.backup.backupPath default + membership ai/configBase.mjs default becomes checkout-independent (${HOME}/.neo-ai/backups) and the leaf is reclassified planeMember: false with a non-empty planeMemberReason, leaving PLANE_MEMBER_PATHS; NEO_BACKUP_PATH still overrides explicit override continues to win; derivePlaneMemberPaths set-equality fails closed if the declared list and the descriptor tree disagree deployment guide backup section unit: default is not a descendant of neoRootDir or of the plane anchor; non-member decision carries a reason; override still honored
Compose host source ai/deploy/docker-compose.yml host bind source is ${NEO_HOST_BACKUP_ROOT:-${HOME}/.neo-ai/backups}; the default does not derive from the Compose project/checkout path (an explicit override is unconstrained) absent env resolves that default Compose profile docs + deployment cookbook static spec asserts the source names the env var and that neither it nor its :- fallback is project-relative as a class — absolute-rooted required, so bare backups / nested/backups fail alongside ./ and ../ — plus exact-match against this ledger's default. Red-proofs recorded for ./-prefixed and bare-relative
Compose container target ai/deploy/docker-compose.yml canonical orchestrator sets NEO_BACKUP_PATH=/app/.neo-ai-data/backups explicitly; parity places its own under the relocated root mandatory, not optional — without it the non-member default resolves to an unbound in-container path and bundles die on recreate Compose profile docs static spec asserts the env entry and mount target, and that host source and target are not collapsed
Compose guarded-key census ai/scripts/lint/config-leaf-parity.json NEO_BACKUP_PATH classified as a required deployment input; census 28 → 29 AiConfig lint fails on census drift lint-config-template-ssot.mjs
ADR 0019 §10.7 matrix row ADR 0019 names the relocated planeMember and its per-profile placement none — an unlisted plane member fails the boot walk ADR 0019 plane-config coherence + assertPlaneMemberCoherence
legacy in-tree bundles ai/scripts/maintenance/backup.mjs one-time notice naming the old path when bundles are detected there silence if none found deployment guide unit: notice fires once when the legacy path is non-empty

Decision Record impact

amends ADR 0019 — relocating a planeMember: true leaf is explicitly a revalidation trigger under its §10.7 election ("moving a profile-pinned leaf … reopens this election"), requiring the matrix update and the coherence re-run named above. No other ADR authority is challenged; ADR 0027's recovery-mutation ownership is untouched because this changes only where bundles are written, never who may consume them.

Acceptance Criteria

  • git clean -xdn on a clean checkout does not list the backup bundle root. This is the headline check and it should be asserted, not eyeballed.
  • backupPath default resolves to a path outside neoRootDir; a unit test asserts the resolved default is not a descendant of the repository root.
  • NEO_BACKUP_PATH override continues to win, with a test proving a deployment that sets it is unaffected.
  • The Compose bind source and the config default resolve to the same location; static Compose placement coverage fails if they diverge.
  • ADR 0019 §10.7 carries the relocated plane member, and plane-config coherence plus assertPlaneMemberCoherence pass.
  • Bundles already present at the legacy in-tree path produce a one-time notice naming that path; nothing is moved or deleted automatically.
  • verifyLatestBackupRestorable returns a coherent verdict against the relocated root — in particular BUNDLE_ROOT_MISSING and NO_BUNDLES remain distinguishable, since restore.mjs:718-729 separates them precisely because of the coupling this ticket removes.
  • Deployment docs state where bundles land and that the location is deliberately outside the checkout.

Out of Scope

  • Off-host or off-machine sync. That is a different guarantee (surviving host loss, not surviving repository operations) and #16199 owns the requirement-enforcement half.
  • Restore-on-boot behavior or its default. #15639 owns that.
  • Retention policy, bundle format, or compression. Retention appears to be working — the 36 bundles span exactly 30 days against maxDays: 30.
  • Migrating existing bundles, beyond emitting the notice in Fix item 4.
  • The disk-exhaustion risk from 133 GB of bundles sharing a disk with the graph. Real, and worth its own ticket, but a separate concern from location.

Avoided Traps

  • Removing .neo-ai-data from .gitignore. The exposure is not caused by the ignore being wrong — it is caused by ignored files living in the tree. Tracking 133 GB would be far worse, and the ignore entry must stay for the tree's other members.
  • Fixing this with documentation. "Do not run git clean -xdf" is not a control. The operator this protects is precisely the one who will not read it, and build-hygiene commands are run reflexively.
  • Relying on the operator's manual out-of-tree copy made 2026-07-30. That protects a snapshot; the default still writes in-tree tomorrow, so the exposure returns with the next bundle.
  • Auto-migrating the bundles as part of the fix. Moving 133 GB implicitly, during a config change, on a machine whose backups are the only copy, is a worse risk than leaving them where a notice can point at them.

Related

  • #16199 — required off-host backup runs exit 0 after a local receipt. Adjacent and complementary: that ticket makes an unmet durability requirement fail; this one makes the bundles survive routine repository operations. Neither subsumes the other.
  • #16167 — the hard cut, whose data-plane replacement is the operation most dependent on a surviving restorable copy
  • #15639 — opt-in first-boot restore, which consumes these bundles
  • #15693 — orchestrator-governed restore-delta-merge
  • #16040 — deployment-guide rewrite, natural consumer of the documentation AC
  • ADR 0019 §10.7 (plane-member placement), ADR 0027 (recovery mutation authority)

Origin Session ID: 0a7f5f1d-cf12-4698-984c-17b64eea5178

Retrieval Hint: backupPath default inside working tree git clean -x removes ignored backups planeDataRootDefault neoRootDir compose bind relative project directory

tobiu referenced in commit 66bce83 - "fix(config): relocate the backup root out of the plane and the working tree (#16201) (#16221) on Jul 31, 2026, 3:57 PM
tobiu closed this issue on Jul 31, 2026, 3:57 PM