LearnNewsExamplesServices
Frontmatter
id10944
titleRemove stale sqlite-vec artifacts from Memory Core healthcheck
stateClosed
labels
bugdocumentationairefactoringarchitecture
assigneesneo-gpt
createdAtMay 8, 2026, 12:06 PM
updatedAtMay 9, 2026, 11:15 PM
githubUrlhttps://github.com/neomjs/neo/issues/10944
authorneo-gpt
commentsCount2
parentIssue10822
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtMay 8, 2026, 12:26 PM

Remove stale sqlite-vec artifacts from Memory Core healthcheck

Closedbugdocumentationairefactoringarchitecture
neo-gpt
neo-gpt commented on May 8, 2026, 12:06 PM

Context

Operator surfaced a healthcheck semantics drift on 2026-05-08: Memory Core now uses ChromaDB for memories and session summaries, while better-sqlite3 backs the Native Edge Graph. The prior sqlite-vec / SQLite-vector alternative was empirically rejected and is no longer a supported path: no HNSW, no skip lists, brute-force vector scans only. There is no backwards-compatibility requirement for that path.

Current healthcheck output still exposes confusing SQLite-vector-era artifacts. Empirical snapshot from @neo-gpt current session:

"database": {
  "connection": {
    "connected": true,
    "collections": {
      "memories": {"name": "neo-agent-memory", "exists": true, "count": 293},
      "summaries": {"name": "neo-agent-sessions", "exists": true, "count": 0}
    },
    "engines": {"chroma": true, "sqlite": false}
  }
},
"migration": {"memory": 0, "session": 0, "total": 0, "available": true}

That reads as if SQLite is an optional/failed database engine beside Chroma, while another block simultaneously proves the SQLite graph is queryable enough for migration counts. The result is misleading operator and agent diagnostics.

Duplicate sweep:

  • #9924 is the closed April two-pillar migration ticket; it established separate Chroma/graph diagnostics before the current sqlite-vec cleanup reality.
  • #10773/#10799 shipped SQLite-side embedding-provider observability, then #10804 consolidated provider selection; remaining docs/comments still carry stale SQLite-side embedding language.
  • #10822 already records sqlite-vec as an avoided dead end and mandates hard cuts, but it does not yet have a concrete healthcheck cleanup sub-ticket.

The Problem

The healthcheck payload and docs still encode stale concepts from a dropped design:

  1. database.connection.engines.sqlite is a permanent false-looking artifact. HealthService.#checkDatabaseConnections() initializes {chroma: false, sqlite: false} and only ever sets engines.chroma = true; it never validates the better-sqlite graph as a peer vector engine.
  2. The providers.embedding docs/comments say the embedding route is used by “ChromaDB retrieval and SQLite Native Edge Graph operations.” That is now misleading: Chroma owns vector memory/session retrieval; better-sqlite owns graph topology.
  3. config.template.mjs still says vectorDimension is enforced “across all SQLite collections,” which points readers back toward the dropped sqlite-vec mental model.
  4. openapi.yaml still documents database.connection.engines.sqlite as a boolean response field.

This creates exactly the failure mode seen during the wake incident investigation: agents over-interpret engines.sqlite=false as a graph-health signal, even though the field is not the source of truth for better-sqlite graph readiness.

The Architectural Reality

  • Memory/session vector storage: ChromaDB collections neo-agent-memory and neo-agent-sessions via StorageRouter / ChromaManager.
  • Graph storage: better-sqlite3-backed Native Edge Graph via ai/graph/storage/SQLite.mjs and GraphService.db.storage.db.
  • Healthcheck source: ai/mcp/server/memory-core/services/HealthService.mjs.
  • Healthcheck schema: ai/mcp/server/memory-core/openapi.yaml.
  • Operator docs: learn/agentos/MemoryCore.md and shared deployment docs.
  • Parent cleanup epic: #10822, especially the “sqlite-vec dead-end; Chroma stays for vectors; better-sqlite stays for graph” constraint.

The Fix

Hard-cut stale sqlite-vec / SQLite-vector artifacts from the Memory Core healthcheck contract and docs.

Recommended shape:

  1. Replace or remove database.connection.engines.sqlite. If graph readiness is needed, expose it under an explicit graph block such as:
"database": {
  "connection": {
    "engines": {"chroma": true},
    "graph": {
      "backend": "better-sqlite3",
      "available": true,
      "path": ".neo-ai-data/sqlite/memory-core-graph.sqlite"
    }
  }
}
  1. Ensure graph availability, if surfaced, comes from the actual better-sqlite graph handle/path, not from retired vector-engine state.
  2. Update openapi.yaml to remove or rename the stale sqlite engine field.
  3. Update learn/agentos/MemoryCore.md and relevant config comments to state the current split clearly: ChromaDB for memories/summaries vectors; better-sqlite3 for Native Edge Graph topology.
  4. Remove stale references to SQLite-side embedding / sqlite-vec compatibility unless they are explicitly in “rejected/dead-end” history sections.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback / Edge Case Docs Evidence
Memory Core healthcheck.database.connection payload This ticket, #10822, current operator correction Chroma connectivity reports the Chroma vector substrate for neo-agent-memory / neo-agent-sessions; graph readiness, if reported, is explicitly named as better-sqlite3 graph readiness, not sqlite vector-engine status. If GraphService is not initialized during cold healthcheck, graph block returns available: false with a clear readiness reason; it must not imply sqlite-vec is supported. learn/agentos/MemoryCore.md healthcheck section, openapi.yaml schema Unit tests assert no stale engines.sqlite: false artifact and assert graph block semantics if added.
providers.embedding docs/comments #10804 consolidated provider selector + operator correction Describe the embedding provider as the Chroma/vector embedding route used for semantic retrieval and ingestion. Do not claim a SQLite vector consumer. Historical sqlite-vec notes may remain only in avoided-trap / dead-end context. HealthService.mjs JSDoc, MemoryCore.md, SharedDeployment.md if applicable Grep test or review evidence: no live healthcheck docs mention SQLite-side embedding except rejected-history sections.
Config vectorDimension comments #10822 hard-cut cleanup Describe vector dimension as Chroma embedding dimension / semantic vector dimension. None; sqlite-vec compatibility is dropped. memory-core/config.template.mjs, any shared config docs Unit/docs grep evidence removes “SQLite collections” phrasing for vector dimensions.

Acceptance Criteria

  • HealthService.#checkDatabaseConnections() no longer emits a misleading engines.sqlite: false placeholder.
  • If graph readiness is surfaced, it is explicitly named as better-sqlite3 / Native Edge Graph readiness and is validated against the actual graph substrate.
  • openapi.yaml reflects the new healthcheck shape and removes the stale SQLite-vector engine implication.
  • learn/agentos/MemoryCore.md explains the current split: ChromaDB stores memory/session vectors; better-sqlite3 stores graph topology.
  • Stale “SQLite-side embedding” / “SQLite collections” language is removed from live config and healthcheck docs, except where explicitly framed as rejected history.
  • Unit coverage for HealthService verifies the corrected response shape.
  • git diff --check origin/dev...HEAD passes before PR.

Out of Scope

  • Reintroducing sqlite-vec, SQLite VSS, or any SQLite vector-store alternative.
  • Backwards compatibility for engines.sqlite; this was a dev/substrate artifact, not a supported public contract.
  • Changing ChromaDB collection names or storage routing.
  • Solving broader config-topology cleanup already tracked under #10822 beyond this healthcheck artifact.

Avoided Traps

  • Keeping engines.sqlite “for compatibility.” Rejected. It now misleads agents and operators, and the underlying sqlite-vec path is fully dropped.
  • Treating better-sqlite graph readiness as a vector-engine readiness flag. Rejected. The graph is topology storage, not a Chroma replacement.
  • Using docs-only cleanup. Rejected. The payload/schema itself creates the confusion, so implementation and schema must change together.

Related

  • Parent epic: #10822
  • Historical predecessor: #9924
  • Stale provider-observability predecessor: #10773 / PR #10799
  • Provider consolidation: #10804
  • SQLite-vec dead-end source trail: Discussion #10819 and #10822 avoided-traps section

Origin Session ID: c02fbf4e-870c-44c0-ba7e-e9ffacce094b

Retrieval Hint: query_raw_memories(query="Memory Core healthcheck sqlite false sqlite-vec dropped Chroma memories sessions better-sqlite graph")

tobiu referenced in commit 98897fc - "feat(ci): re-add unit suite to matrix post-Bucket-G substrate (#10939) (#10953) on May 8, 2026, 2:43 PM