Chroma: one flat unified store, identical local + cloud

Context

The #12142/#12152 defrag-path work surfaced that the Chroma persistence topology has drifted off-piste, and a deep audit (Origin Session below) confirmed it. The live local store is one daemon serving a folder named knowledge-base that actually holds all realms — neo-knowledge-base (26,405 vecs), neo-agent-memory (14,722), neo-agent-sessions (1,075), neo-native-graph (612) — plus a stale chroma/memory-core/ folder (pre-ADR-0003 federated leftover, ~11k stale vecs, last written 2026-05-16) and ~1,134 leaked test-* collections.

Meanwhile the cloud deployment (ai/deploy/docker-compose.yml) already runs one chroma container with one flat volume (chroma-data:/chroma/chroma). So local (two folders, --path …/knowledge-base) and cloud (one flat volume) have diverged — there is no dev/prod parity. Operator decision: one flat store everywhere.

The Problem

1. Misleading name. The persist dir named knowledge-base holds every realm — it reads as "MC stores into the KB." #12152 enshrined engines.chroma.dataDir = …/knowledge-base as the SSOT, codifying the confusion.
2. Tech debt. The stale chroma/memory-core/ folder is a federated-era leftover; the live data is in the served store. It does not exist in cloud at all.
3. Privacy landmine. ai/scripts/maintenance/uploadKnowledgeBase.mjs runs zip -r neo-ai-data.zip .neo-ai-data — the whole private tree (199 MB memory-core-graph.sqlite, all memories, mailbox). downloadKnowledgeBase.mjs extracts that over .neo-ai-data — which would clobber a user's MC. Currently latent (the shipped v12.1.0 asset is the clean 2.2 MB chroma-neo-knowledge-base.zip; the script drifted to .neo-ai-data on 2026-04-06 and the asset name no longer matches → the path is effectively unreleased/broken). The unification turned the once-clean "zip the KB folder" into a leak.
4. No parity. Local two-folder layout vs cloud one-flat-volume.

The Architectural Reality

• ChromaDB: one daemon ⟺ one persist --path; HNSW indices are per-collection / per-segment — coexistence in one store does not degrade per-collection search latency/recall. The UUID dirs under the persist root are the independent hnswlib indices.
• Cloud (docker-compose.yml): one chroma container (chromadb/chroma:1.5.9), chroma-data:/chroma/chroma; kb-server + mc-server + orchestrator connect via NEO_CHROMA_HOST=chroma / NEO_CHROMA_PORT=8000; NEO_ORCHESTRATOR_CHROMA_DAEMON_ENABLED=false (the container owns the daemon). Local: the orchestrator launches chroma run --path .neo-ai-data/chroma/knowledge-base.
• Isolation = collection names + metadata, not folders (learn/agentos/cloud-deployment/Overview.md: "tenant isolation is enforced by the identity tuple + write-side stamping + read-side filter, not by physical separation"). MC connects host:port; dataDir is consumed only by the on-disk maintenance scripts.
• Recovery model: KB = rebuildable cache (ai:sync-kb); MC = irreplaceable store (backup/restore).

The Decision (locked)

One flat Chroma store named unified, identical shape local and cloud — local .neo-ai-data/chroma/unified, cloud volume mounted at /chroma/unified (retiring the confusing /chroma/chroma). One daemon/container, no per-realm folders. KB / MC / tenant separation is enforced at the layers that exist everywhere — collection names + metadata (isolation), collection-scoped export (shipping), KB-as-cache / MC-as-store backup model (recovery) — never by directory or daemon split. Local conforms to the cloud reference; the two-folder layout and the stale memory-core folder are retired.

Decision Record impact

amends ADR 0003 (Chroma Topology — Unified Only): preserves its one-daemon core decision; adds the flat-store naming, the local↔cloud parity mandate, and the separation-layer model; retires the federated-leftover local two-folder layout. Touches ADR 0014 (Cloud Deployment Topology).

ADR successor-risk: adr-amendment-required — artifact this epic; ADR 0003 Accepted 2026-05-09; evidence (docker-compose.yml one-container + flat volume; uploadKnowledgeBase.mjs whole-dir zip; live store collection inventory; Overview.md topology anchor); route amendment (a new ADR that amends 0003), not silent bypass.

Contract Ledger (cross-cutting surfaces)

Sub-tickets (work breakdown)

1. Amending ADR — author the decision record (one flat unified store; parity; separation layers) amending ADR 0003 + touching ADR 0014.
2. Rename + migrate (core) — SSOT value → …/chroma/unified; orchestrator --path → unified; cloud compose volume → /chroma/unified + persist-dir; one-time migration (move data, delete stale memory-core). Builds on #12152.
3. defragChromaDB.mjs — two TARGETS → two collection-groups over the one unified store; keep the #12140 whole-store keep-set.
4. Release-pipeline privacy fix ⚠️ (highest priority — latent leak) — uploadKnowledgeBase.mjs collection-scoped export of neo-knowledge-base only (never zip -r .neo-ai-data); downloadKnowledgeBase.mjs collection-scoped import (never dir-restore); publish.mjs wiring; reconcile the neo-ai-data.zip vs chroma-neo-knowledge-base.zip name drift.
5. Guide updates — learn/agentos/cloud-deployment/* (Overview, Day0Tutorial, Configuration) + defrag/backup docs.

Companions (existing)

• #12152 — the engines.chroma.dataDir SSOT foundation (keeper; sub-task 1 / merge precedes the rename).
• #12143 — test-pollution leak → ephemeral test store (required so we never ship/defrag test-*).
• #12139 — orchestrator owns the one daemon, servers as pure external clients (aligned; nests here).

Acceptance Criteria

• Amending ADR merged; ADR 0003 + 0014 cross-referenced.
• One flat store unified local + cloud; no knowledge-base/memory-core subfolders; stale memory-core deleted.
• defrag operates per-collection-group on the one store (KB-safe AND MC-safe; #12140 keep-set spans the store).
• KB release artifact is collection-scoped — never ships MC collections; user ingest never clobbers MC. (end-to-end = post-merge validation)
• #12143 test-pollution removed; tests use an ephemeral store.
• Guides updated; HealthService topology: 'unified' still accurate.

Out of Scope

• Multi-tenant KB-collection HNSW scaling — the one neo-knowledge-base collection holds all tenants (metadata-filtered); if that becomes a perf concern it's a separate axis, not this epic.
• Two-daemon / federated topology — explicitly rejected (cloud one-container forbids it).

Avoided Traps

• Two daemons / two live folders (an earlier proposal in the originating analysis) — structurally impossible under the cloud one-container model; would break dev/prod parity.
• Folder-level KB shipping — unsafe post-unification (would leak MC); collection-scoped export is the only correct mechanism and works identically in cloud (where no KB folder exists).

Related

#12152, #12143, #12139, #12142; ADR 0003, ADR 0014.

Origin Session ID: efd8dc2e-2052-4089-814a-ab22cd8c6a62 Retrieval Hint: "one flat unified chroma store local cloud parity rename knowledge-base memory-core"