LearnNewsExamplesServices
Frontmatter
id14031
titleCaller-keyed Chroma write-guard (config-independent test-isolation)
stateClosed
labels
enhancementaitestingarchitecture
assigneesneo-opus-vega
createdAtJun 26, 2026, 1:20 AM
updatedAtJun 26, 2026, 9:32 AM
githubUrlhttps://github.com/neomjs/neo/issues/14031
authorneo-opus-vega
commentsCount0
parentIssue14039
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJun 26, 2026, 9:32 AM
milestonev13.1

Caller-keyed Chroma write-guard (config-independent test-isolation)

Closed v13.1.0/archive-v13-1-0-chunk-6 enhancementaitestingarchitecture
neo-opus-vega
neo-opus-vega commented on Jun 26, 2026, 1:20 AM

Context

The #14022 physical Chroma test-isolation (separate daemon + data dir under UNIT_TEST_MODE) is the right primary containment, but it is config-resolution-dependent: during the #14022 review, running the slice with a stale local config overlay made the active engines.chroma.dataDir resolve to the production path under UNIT_TEST_MODE — i.e. isolation silently fell back to prod coords. The same exposure applies to any run that doesn't load the unit config (a bare npx playwright — the documented 2026-05-17 wipe vector).

The Problem

Config-driven isolation is only as strong as config resolution. When it goes wrong (stale overlay, missed migration, no-config run), unit-test Chroma writes silently target the live store — the exact bleed class. The graph (SQLite) and concept-ontology paths already have a config-independent caller-keyed guard; the Chroma write path does not.

The Architectural Reality

assertTestWriteIsolated (fires on TEST_WORKER_INDEX — set by Playwright per worker — regardless of UNIT_TEST_MODE/config) is wired into:

  • ai/graph/storage/SQLite.mjs (5 write funnels, #13639)
  • ai/services/ingestion/ConceptDiscoveryService.mjs via ai/services/shared/storeWriteGuard.mjs (#13683)

It is NOT wired into the Chroma write path. The destructive-delete guard (#11652 chromaDeleteCollection / ChromaManager.guardedDeleteCollection) covers deletes-by-target, but additive writes have no caller-keyed gate.

The Fix

Add a caller-keyed assertTestWriteIsolated-equivalent to the Chroma write path: when a test runner is detected (TEST_WORKER_INDEX / UNIT_TEST_MODE) AND the resolved Chroma coords are production-like, throw before the write — config-independent, fires regardless of overlay/config state. Symmetric to #13639 / #13683.

Re-scope (PR #14044 cycle-1/N review — the broad caller-keyed coord predicate is UNACHIEVABLE)

The original "block any test-caller + production-like coords" predicate (above) is not achievable at the Chroma write/coordinate layer — proven during the #14044 review, cross-confirmed by @neo-gpt's coordinate probe. A fresh-workspace / cloud orchestrator daemon is a legitimate test-caller context: it inherits Playwright's TEST_WORKER_INDEX and resolves the full production coordinate tuple (dataDirProd is cwd-independent, so host/port/dataDir/database all = prod) while writing production-named collections (neo-agent-memory). It is byte-indistinguishable from a bleed at the coord layer; a caller-keyed coord guard crashes it at boot (the cycle-1 integration-unified regression).

The achievable discriminator is the collection NAME: the #14010 bleed creates test-* collections (test-memory-* / test-session-*) in the production database; a legitimate prod-coordinate daemon creates neo-* collections. So the guard fails closed on a test-* name resolving into the production database. The broad "prod-named write under a test caller" class is covered by #14022 physical isolation (separate daemon/dataDir prevents a correctly-configured test from reaching prod coords), not a coord guard — there is no achievable coord-level guard for it, so it is NOT a permanently-open AC here.

Acceptance Criteria (revised — achievable guard)

  • A test-*-named collection (test-memory-* / test-session-*) resolving into the production database (default_database) fails closed — the #14010 bleed signature, at the collection-creation boundary.
  • No fresh-workspace false-positive: neo-*-named collections in production coordinates proceed (the legitimate cloud/fresh-workspace daemon — inherited TEST_WORKER_INDEX must not crash it).
  • Unit coverage: test-* + prod DB rejected; neo-* + prod DB allowed; test-* + test DB allowed.
  • The broad "prod-named write under a test caller" class is documented as covered by #14022 physical isolation (not this name guard).

Out of Scope

  • The #14022 physical isolation (this is its config-independent backstop, fast-follow sibling — not a replacement).
  • The "tests must always use config.template" issue (separately ticketed per operator).

Related

  • #14022 — physical isolation (this backstops its fallback case).
  • #13639 — the SQLite caller-keyed guard this mirrors.
  • #13683 — the concept-ontology caller-keyed guard.
  • #14010 — the test-bleed regression.

Origin Session ID: 58acd7a6-778e-4fe6-b85a-4569802ac57b

Handoff Retrieval Hints: query_raw_memories("caller-keyed Chroma write guard assertTestWriteIsolated TEST_WORKER_INDEX config-independent"); anchors: ai/services/shared/storeWriteGuard.mjs, ai/services/shared/vector/chromaClientPrimitives.mjs, ai/graph/storage/SQLite.mjs:322.

Authored-by: @neo-opus-vega (Vega, Claude Opus 4.8)

tobiu referenced in commit a3aae2a - "fix(ai): config-independent Chroma test-write guard (#14031) (#14044) on Jun 26, 2026, 9:32 AM
tobiu closed this issue on Jun 26, 2026, 9:32 AM