LearnNewsExamplesServices
Frontmatter
id11981
title[Sub-2 of #11976] Add KB test-fixture (KB_DEFAULTS) + migrate SourcePathsConfig drift-vulnerable assertion
stateClosed
labels
enhancementaitesting
assigneesneo-opus-4-7
createdAtMay 25, 2026, 4:43 PM
updatedAtMay 26, 2026, 1:39 AM
githubUrlhttps://github.com/neomjs/neo/issues/11981
authorneo-opus-4-7
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtMay 26, 2026, 1:39 AM

[Sub-2 of #11976] Add KB test-fixture (KB_DEFAULTS) + migrate SourcePathsConfig drift-vulnerable assertion

Closedenhancementaitesting
neo-opus-4-7
neo-opus-4-7 commented on May 25, 2026, 4:43 PM

Sub-2 of #11976 — per-server KB fixture establishes the pattern from Sub-1 (#11977/#11978) at the per-server tier.

V-B-A scope correction

The umbrella estimated 18 KB config.mjs imports as candidates. V-B-A on actual usage patterns found only 1 spec is drift-vulnerable; the other 17 are mutation-style or config-as-source-of-truth (round-trip; drift-immune by design):

Spec Pattern Action
SourcePathsConfig.spec.mjs expect(aiConfig.sourcePaths?.X ?? 'default').toBe('default') MIGRATE (drift-vulnerable: customizing sourcePaths.X breaks the assertion)
AdrSource.spec.mjs aiConfig.neoRootDir = mockRoot; ... = originalRoot Mutation; keep
ChromaManager.spec.mjs expect(collection.name).toBe(aiConfig.collectionName) Round-trip; keep
SearchService.spec.mjs path.resolve(aiConfig.neoRootDir, 'tmp', ...) Setup-wiring; keep
logger.spec.mjs aiConfig.data.logPath = tmpLogDir; ... = originalLogPath Mutation; keep
(13 others) Same patterns — mutation/setup-wiring/round-trip Keep

The umbrella's "65 drift-vulnerable imports" estimate was an overcount. The actual count across all server-groups is likely much lower; the per-server fixtures still have value as durable substrate for future assertion-style tests.

The Fix

  1. New fixture test/playwright/fixtures/knowledgeBaseConfigDefaults.mjs:

    • Inlines the KB template's canonical default values as a deep-frozen snapshot (mirrors aiConfigDefaults.mjs's inline pattern from Sub-1 cycle-4 — not an import of knowledge-base/config.template.mjs). Inlining sidesteps the class-registration ordering hazard: importing the template registers Neo.ai.mcp.server.knowledge-base.Config at fixture-import time, which collides with specs (like SourcePathsConfig.spec.mjs) that override the singleton via per-test delete/restore patterns. Plain-data inline is the correct Sub-1 substrate shape applied at the per-server tier.
    • Exports KB_DEFAULTS = deepFreeze(Neo.clone({...inline KB-template defaults...}, true, true)) — uses Neo's canonical deep-clone primitive per #11978 cycle-4.
    • Does NOT re-export KBConfig singleton. V-B-A on the 17 mutation-style specs confirmed they already import the singleton directly via (await import('.../config.template.mjs')).default. Adding a fixture-side re-export would create a second import path for the same singleton (two-source-of-truth anti-pattern). The fixture's exclusive surface is the immutable snapshot for assertion-only use.
    • JSDoc explains the per-server vs Tier-1 boundary: assertions on Tier-1 fields (auth, engines.chroma, modelProvider) should use TIER1_DEFAULTS from aiConfigDefaults.mjs; KB_DEFAULTS is the right surface for KB-template-defined fields (collectionName, sourcePaths, path, etc.).
  2. Migrate SourcePathsConfig.spec.mjs:

    • Add import {KB_DEFAULTS} from '.../test/playwright/fixtures/knowledgeBaseConfigDefaults.mjs'
    • Update the drift-vulnerable LearningSource base-directory assertion at line 310 to compare against KB_DEFAULTS.sourcePaths?.LearningSource instead of the live overlay-merged aiConfig.sourcePaths?.LearningSource
    • 17 other KB specs: NOT migrated (correct usage patterns, per V-B-A audit)

Acceptance Criteria

  • test/playwright/fixtures/knowledgeBaseConfigDefaults.mjs exists with KB_DEFAULTS export (inline frozen snapshot via Neo.clone) + JSDoc explaining the per-server vs Tier-1 boundary + the inline-vs-template-import rationale.
  • SourcePathsConfig.spec.mjs LearningSource base-directory test migrated to use KB_DEFAULTS; assertion is now overlay-immune.
  • The 17 non-drift-vulnerable KB specs are left untouched.
  • All affected specs PASS (focused KB suite + the Sub-1 worker-collision scenario from #11969).
  • Operator-side smoke: customize ai/mcp/server/knowledge-base/config.mjs sourcePaths.AdrSource; SourcePathsConfig spec still passes against the fixture defaults.

Out of Scope

  • The other 17 KB specs — their patterns are correctly drift-immune; migrating would be churn without benefit.
  • KBConfig live-singleton re-export from the fixture — 17 mutation-style specs already import the singleton directly; second import path = two-source-of-truth anti-pattern.
  • Per-server fixture for the Tier-1 sub-fields (auth, engines.chroma) — those are TIER1_DEFAULTS's territory; per-server fixtures cover per-server-specific fields only.
  • GW (Sub-3), NL (Sub-4), MC (Sub-5) — separate subs.

Out of Scope (umbrella decomposition correction)

Original umbrella estimated GW (8), NL (4), MC (46) drift-vulnerable. Those numbers may also be overcount — Subs 3/4/5 should each run the same V-B-A audit before committing scope.

Related

Handoff Retrieval Hint: "Sub-2 11976 KB_DEFAULTS knowledge-base test-fixture 1-spec migration scope correction".

tobiu referenced in commit 2949ef9 - "feat(test-fixtures): add KB_DEFAULTS + migrate SourcePathsConfig drift-vulnerable assertion (#11981) (#12001) on May 26, 2026, 1:39 AM
tobiu closed this issue on May 26, 2026, 1:39 AM