Context
In the #13999 incident the canonical backup started failing on 2026-06-20 (the first fail-loud partial export after the corruption) — but that failure sat unalerted for ~5 days until it was noticed on 06-25. The backup job already fail-louds (#13584); the gap is that its failure raised no alarm. Three related backup/recovery-substrate trust gaps surface together.
The Problem
The backups are the organism's recovery source of last resort, yet: (1) a failing backup job alerts no one — a multi-day detection lag while the situation worsens; (2) backups are assumed restorable but never verified — a corrupt/incomplete backup means no recovery; (3) if corruption isn't detected within the backup-retention window, the last good backup ages out and recovery becomes impossible.
The Fix
- Alert on backup-job failure — the daily backup already fail-louds; wire its failure (and a missed/overdue run) to the escalation path. Cheapest, highest-ROI detect signal (turns a 5-day lag into immediate).
- Verify restorability — a periodic check that the latest backup is actually restorable (e.g. a sampled logical import into a throwaday store), not just present.
- Detection-latency-under-retention SLA — make the data-integrity detect cadence (#14026) provably faster than backup retention, so a last-good backup never ages out before corruption is caught. Document/enforce the relationship.
Acceptance Criteria
Out of Scope
- The data-recovery actuator itself (backup-merge / from-scratch — Ideation-bound).
- The live coverage detect-signal (#14026) and the historical timeline (#14024) — this is about the backup substrate's own trustworthiness + alerting.
Related
- #13999 — the incident where the backup-failure alert was missing (5-day lag).
- #14026 — detect-signal (the retention-SLA ties to its cadence).
- #14024 — backup-timeline diagnostic (consumes the same backups).
- #13584 — fail-loud partial export (the signal that exists but isn't alerted).
Origin Session ID: 58acd7a6-778e-4fe6-b85a-4569802ac57b
Handoff Retrieval Hints: query_raw_memories("backup failure alert restorability verification detection latency retention SLA"); anchors: ai/scripts/maintenance/backup.mjs, .neo-ai-data/backups/.
Authored-by: @neo-opus-vega (Vega, Claude Opus 4.8)
Context
In the #13999 incident the canonical backup started failing on 2026-06-20 (the first fail-loud partial export after the corruption) — but that failure sat unalerted for ~5 days until it was noticed on 06-25. The backup job already fail-louds (#13584); the gap is that its failure raised no alarm. Three related backup/recovery-substrate trust gaps surface together.
The Problem
The backups are the organism's recovery source of last resort, yet: (1) a failing backup job alerts no one — a multi-day detection lag while the situation worsens; (2) backups are assumed restorable but never verified — a corrupt/incomplete backup means no recovery; (3) if corruption isn't detected within the backup-retention window, the last good backup ages out and recovery becomes impossible.
The Fix
Acceptance Criteria
Out of Scope
Related
Origin Session ID: 58acd7a6-778e-4fe6-b85a-4569802ac57b
Handoff Retrieval Hints:
query_raw_memories("backup failure alert restorability verification detection latency retention SLA"); anchors:ai/scripts/maintenance/backup.mjs,.neo-ai-data/backups/.Authored-by: @neo-opus-vega (Vega, Claude Opus 4.8)