Restoration Runbook
This document provides consistent recovery procedures for all persistent subsystems within the Neo.mjs AI substrate, leveraging the atomic bundles generated by the Daily Snapshot Pipeline (npm run ai:backup).
Bundle Layout Overview
Backups are stored in .neo-ai-data/backups/backup-<timestamp>/ and contain the following atomic directories:
kb/: Knowledge Base ChromaDB as JSONLmc/: Memory Core memories and summaries as JSONLgraph/: Memory Core SQLite graph as JSONLconcepts/: Concept Ontology JSONLtrajectories/: RLAIF training trajectories JSONL
Prerequisites
Before initiating any restoration, ensure that all AI MCP servers and daemon processes are stopped (terminate any running npm run ai:server processes).
Restoration Procedures
1. Knowledge Base (KB)
The Knowledge Base is the neo-knowledge-base collection inside the one flat unified ChromaDB store (ADR 0017) — a cache, not a store. Recover it by deterministic rebuild from source, at collection scope. Never delete the store folder: chroma/unified also holds the irreplaceable Memory Core collections.
Procedure:
- Re-synchronize the KB from the source files (deterministic rebuild — clears and repopulates only the
neo-knowledge-basecollection):npm run ai:sync-kb
(Note: Direct JSONL import of thekb/bundle is deferred to #10871. Do notrm -rfthechroma/unifiedfolder — it is shared with Memory Core; KB recovery is collection-scoped, handled by the rebuild above.)
2. Memory Core (MC) - Memories & Summaries
Memory Core memories and session summaries live as the neo-agent-memory and neo-agent-sessions collections inside the same flat unified ChromaDB store (ADR 0017). Unlike the KB, MC is the irreplaceable store — recover it from the backup bundle, at collection scope via the SDK. The pre-unification chroma/memory-core/ folder is retired.
Procedure:
- Re-import the MC JSONL from the backup bundle via the SDK (
mode: 'replace'clears and repopulates the MC collections at collection scope — no folder deletion):node -e "import('./ai/services.mjs').then(s => s.default.memory.manageDatabaseBackup({action: 'import', file: '.neo-ai-data/backups/backup-<timestamp>/mc/memory-backup-<timestamp>.jsonl', mode: 'replace'}))"(Note: A dedicatedrestore.mjsCLI is deferred to #10871. Do notrm -rfthechroma/unifiedfolder — it is shared with the Knowledge Base; MC restore is collection-scoped via the SDK above.)
3. Memory Core - Native Edge Graph
The Memory Core Edge Graph is persisted in SQLite.
Procedure:
- Move the corrupted SQLite database aside:
mv .neo-ai-data/sqlite/memory-core-graph.sqlite .neo-ai-data/sqlite/memory-core-graph.sqlite.bak - Re-import the Graph JSONL from the backup bundle via the SDK:
node -e "import('./ai/services.mjs').then(s => s.default.memory.manageDatabaseBackup({action: 'import', file: '.neo-ai-data/backups/backup-<timestamp>/graph/graph-backup-<timestamp>.jsonl', mode: 'replace'}))"
4. Concept Ontology
The Concept Ontology consists of nodes and edges defined in JSONL.
Procedure:
- Clear the active concepts directory:
rm -rf .neo-ai-data/concepts/* - Copy the concepts from the backup bundle:
cp -r .neo-ai-data/backups/backup-<timestamp>/concepts/* .neo-ai-data/concepts/
5. RLAIF Trajectories
The RLAIF trajectories capture interaction feedback and metadata for offline RL alignment.
Procedure:
- Replace the active trajectories file:
cp .neo-ai-data/backups/backup-<timestamp>/trajectories/trajectories.jsonl .neo-ai-data/datasets/rlaif/trajectories.jsonl
Verification
After restoration, restart the subsystem and verify health.
npm run ai:server
Validate recovery by ensuring healthcheck.backup.lastSuccessful surfaces your desired timestamp and no subsystem connection errors are reported.
Important: Never perform destructive restores on shared multi-tenant databases without confirming coordinate isolation. Only restore into intended environments.