LearnNewsExamplesServices
Frontmatter
id12435
titleaiConfig Provider singleton mutated without restore across ~11 test setups — add snapshot/restore isolation
stateClosed
labels
enhancementaitesting
assigneesneo-opus-grace
createdAtJun 3, 2026, 12:41 PM
updatedAtJul 2, 2026, 8:46 AM
githubUrlhttps://github.com/neomjs/neo/issues/12435
authorneo-opus-ada
commentsCount20
parentIssue12456
subIssues
13519 aiConfig snapshot/restore test-isolation helper (AC1 of #12435)
subIssuesCompleted1
subIssuesTotal1
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[x] 13639 Guard graph-node inserts outside UNIT_TEST_MODE (inserts have no guard twin)
closedAtJul 2, 2026, 8:46 AM

aiConfig Provider singleton mutated without restore across ~11 test setups — add snapshot/restore isolation

Closed v13.1.0/archive-v13-1-0-chunk-1 enhancementaitesting
neo-opus-ada
neo-opus-ada commented on Jun 3, 2026, 12:41 PM

Context

PR #12423 review surfaced (operator-flagged) a widespread test anti-pattern: specs mutate the aiConfig Provider singleton in their setup blocks without restoring it. aiConfig is imported once per module graph and mutations stick; dynamic re-imports do not yield a clean instance. This is a latent cross-test corruption hazard.

The Problem

~11 specs write aiConfig.storagePaths.graph (and .autoIngestFileSystem, .engines.chroma.database) in beforeEach / describe-setup without restore, e.g.: GraphService.spec, MC ChromaManager.spec, McpServerToolLimits.spec, WriteSideInvariant.spec, SessionSummarization.spec, MC Server.spec, SessionService.spec / SessionService.ResumeValidation.spec, MemorySessionIngestor.spec, ConceptIngestor.spec, QueryReRanker.spec, DreamService.spec.

Single-spec and parallel (--workers>1, per-file isolation) runs mask the collision; a full-suite --workers=1 run is where the singleton persists across files and one spec's config bleeds into the next. CI's per-file isolation + retries hide it — this is exactly the class that passes CI but is latently wrong.

The Architectural Reality

  • aiConfig is a Neo Provider singleton (one instance per module graph; not reconstructed by re-import).
  • Correct precedent in-tree: test/playwright/unit/ai/mcp/server/shared/services/DestructiveOperationGuard.spec.mjs:223-234 captures originalGraphPath = aiConfig.storagePaths.graph, mutates, and restores it in a finally — the capture/restore pattern the other specs lack.

The Fix (lead prescription)

A shared test-isolation helper that snapshots the mutated aiConfig keys at setup and restores them in afterEach/afterAll (capture/restore), adopted across the ~11 specs. (Alternative shapes — a fresh per-test config Provider, or a scoped-override API — can be weighed during implementation; the snapshot/restore helper is the minimal, precedent-backed fix.)

Acceptance Criteria

  • A shared helper snapshots + restores the mutated aiConfig keys around each affected spec.
  • All enumerated specs adopt it; no unrestored aiConfig.* writes remain in their setup.
  • Negative isolation test: a representative full-suite --workers=1 run is green with no cross-spec config bleed.

Out of Scope

  • Redesigning aiConfig Provider semantics.
  • The #12423 chunking-semantic-soundness concern (separate; design-space → Ideation).

Decision Record impact

none.

Related

  • Surfaced in: PR #12423 review.
  • Correct-pattern precedent: DestructiveOperationGuard.spec.

Origin Session ID: 4cfb33e9-9d3a-4af7-afa8-e3d71ed4bcc7 Retrieval Hint: "aiConfig Provider singleton test mutation no restore cross-test pollution --workers=1 snapshot restore helper #12423"

tobiu referenced in commit bf9cfa4 - "test(ai): by-construction aiConfig isolation for 5 memory-core specs (#12686) (#12687) on Jun 7, 2026, 4:58 PM
tobiu referenced in commit 21d502e - "fix(agentos): B4-safe debug activation via setEnvOverride (#12889) (#12952) on Jun 12, 2026, 5:42 AM
tobiu referenced in commit a3380ac - "feat(ai): fail-build lint banning Class-A DB-path AiConfig mutation in tests (#13227) (#13228) on Jun 14, 2026, 3:36 PM
tobiu referenced in commit 4dcab5e - "test(memory-core): add aiConfig snapshot/restore test-isolation helper (#13315) (#13316) on Jun 15, 2026, 2:02 PM
tobiu referenced in commit 60c1be1 - "feat(ai): per-agent modelProvider resolves via the AiConfig SSOT (#13525) (#13526) on Jun 19, 2026, 10:19 AM
tobiu referenced in commit 62c7a29 - "feat(ai): shared store-write guard util + SQLite adoption (#13665) (#13671) on Jun 20, 2026, 11:31 PM
tobiu referenced in commit 60e87be - "fix(ai): branch handoffFilePath under UNIT_TEST_MODE so test runs don't clobber the tracked handoff (#13663) (#13719) on Jun 21, 2026, 11:31 AM