LearnNewsExamplesServices
Frontmatter
id12143
titleTest runs leak test-* collections into the production unified Chroma store
stateClosed
labels
bugaitesting
assignees[]
createdAtMay 28, 2026, 7:58 PM
updatedAtJun 6, 2026, 1:26 AM
githubUrlhttps://github.com/neomjs/neo/issues/12143
authorneo-opus-ada
commentsCount7
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[x] 12331 Isolate unit-test chroma + sqlite-daemon stores + orphan-purge
blocking[]
closedAtJun 6, 2026, 1:26 AM

Test runs leak test-* collections into the production unified Chroma store

neo-opus-ada
neo-opus-ada commented on May 28, 2026, 7:58 PM

Context

Per ADR 0003 (learn/agentos/decisions/0003-chroma-topology-unified-only.md) the production Chroma store is a single unified persist dir served by one daemon. Playwright unit tests exercise Chroma via the ai/services.mjs SDK and create collections named like test-memory-${pid}-${ts} (e.g. in test/playwright/unit/ai/.../DatabaseService.importMergeChroma.spec.mjs).

Problem

These test-* collections are created against the SAME daemon / persist dir that serves production KB + MC. Some specs add cleanupChromaManager in afterEach, but that cleanup is insufficient in two cases:

  1. Crashed / aborted runs never reach afterEach, leaking their collections.
  2. npx playwright bypass loses the UNIT_TEST_MODE injection that npm run test-unit provides, so destructive / cleanup fixtures don't run against the intended namespace.

Result: orphaned test-* collections accumulate in the production unified store. A prior live-store reading observed on the order of ~1100 such collections (approximate — exact current count to be re-measured; see AC1).

Architectural Reality

afterEach-cleanup against the prod store is the wrong layer: it can only clean the happy path. The robust boundary is STORE-level isolation — test Chroma state should live in an ephemeral store that is torn down wholesale per run, so neither a crash nor an npx bypass can touch the prod persist dir.

Prescription (single)

Isolate test Chroma state from the production unified persist dir: route test collections to a separate ephemeral store (separate persist dir / daemon, created and torn down per run) rather than relying on per-spec afterEach cleanup against the prod store. Then one-time-purge the already-leaked test-* collections from the prod store.

Acceptance Criteria

  1. Measure and report the current count of test-*-prefixed collections in the production unified store (replaces the approximate ~1100).
  2. Test Chroma state resolves to a persist location distinct from the production unified dir (statically verifiable), so that by construction no test run — including a crashed or npx-bypassed one — can create collections in the prod store.
  3. The existing leaked test-* collections are purged from the prod store (one-time janitorial step), verified by a follow-up count of 0.
  4. The isolation mechanism does not regress npm run test-unit behavior or the existing destructive-fixture guards.

Out of Scope

  • The #12140 segment-registry keep-set and the #12138 recycle chain (separate).
  • Any production data migration beyond removing test-*-prefixed collections.

Related

  • Sibling defrag work: #12140 / PR #12141.
  • Harness guard: npm run test-unit vs npx playwright UNIT_TEST_MODE injection.
  • ADR: learn/agentos/decisions/0003-chroma-topology-unified-only.md.

Origin Session ID: f6a4a820-1d95-40e7-910f-b47ad68f51f8