Tests import config.mjs (operator-overlay) instead of config.template.mjs (canonical defaults) — drift risk across ~78 imports

Context

Operator-flagged architectural concern surfaced 2026-05-25 during the session that landed #11969 (the corrective config-overlay-import fix):

i have one more item: inside our ai folder, we want to use /config.mjs . however, i found 86 occurrences for config files inside our test folder. this is dangerous: gitignored files, users can change values, which could then break our tests. challenge: would it be smarter to use config.template files for our tests?

The Problem

After #11969, runtime code correctly imports from the operator-overlay ai/config.mjs (gitignored, auto-created from template via initServerConfigs.mjs). However, the test suite still imports the same operator-overlay files. This is a determinism / CI-vs-local-parity hazard:

• CI: fresh npm install runs prepare, which copies templates to config.mjs files → tests see template-default values.
• Local: operator customizes ai/config.mjs (e.g., sets modelProvider: 'ollama') → tests assert on 'gemini' default, fail locally, pass in CI.

Audit results from grep -rn "config\.mjs" test/ --include="*.mjs":

• 78 import-style references across the test suite (the broader 118-line count includes path-strings in comments/strings).

The actual drift surface is ~65 imports (excluding the /tmp fixtures).

The Architectural Reality

Three usage patterns across the 65 imports:

1. Mutation-style (aiConfig.storagePaths.graph = testDbPath) — writes to the reactive Neo.ai.Config singleton. Template vs mjs both target the same singleton (per the idempotent Neo.setupClass from #11969). No drift risk.
2. Assertion-style (expect(config.modelProvider).toBe('gemini')) — reads default values for comparison. This is the drift-vulnerable surface.
3. Setup-wiring (aiConfig.autoIngestFileSystem = false to pin a known state before a test) — hybrid pattern; immune to overlay because it explicitly sets the value.

Second-order concern from #11969: per-server templates currently import AiConfig from '../../../config.mjs' (the top-level overlay). So even if a test imports the per-server template, it transitively reads operator-overlay values for Tier-1 fields. A pure suffix-swap doesn't close the leak.

The Fix — Option C (test-fixture file)

Recommended architecture: a dedicated test-fixture layer that decouples the assert/mutation concerns:

// test/playwright/fixtures/aiConfigDefaults.mjs (NEW)

// Frozen snapshot of Tier-1 defaults. Tests that need to ASSERT on default
// values should compare against this, not against the live `aiConfig` instance
// (which may carry operator overlays from `ai/config.mjs`).
import AiConfig from '../../../ai/config.template.mjs';

export const TIER1_DEFAULTS = Object.freeze({...AiConfig.data});

// Re-export for tests that explicitly want the live singleton (mutation case).
// Same Neo.ai.Config instance any runtime code sees, courtesy of the idempotent
// setupClass registration from PR #11969.
export {AiConfig};

Per-server analog: test/playwright/fixtures/memoryCoreDefaults.mjs, knowledgeBaseDefaults.mjs, etc., each re-exporting from their respective config.template.mjs.

Test migration pattern:

// BEFORE (drift-vulnerable):
import aiConfig from '../../../../../../ai/mcp/server/memory-core/config.mjs';
expect(aiConfig.modelProvider).toBe('gemini');

// AFTER (deterministic):
import {MEMORY_CORE_DEFAULTS} from '../../../../../../test/playwright/fixtures/memoryCoreDefaults.mjs';
expect(MEMORY_CORE_DEFAULTS.modelProvider).toBe('gemini');

Mutation-style imports stay as-is (no drift risk).

Why Option C over the alternatives

Other options considered and rejected:

• Option A — bulk-swap mjs → template in test imports: doesn't close the second-order leak (per-server templates import top-level mjs). Half-fix.
• Option B — rewire per-server templates to import top-level template: breaks the runtime overlay chain (#11969 fix would regress). Anti-pattern.
• Option D — pin overrides in beforeAll of each test: requires per-test mutation knowledge; doesn't scale; brittle.

Option C separates the concerns cleanly: runtime untouched, tests get a deterministic fixture, no overlay chain leak.

Acceptance Criteria

• test/playwright/fixtures/aiConfigDefaults.mjs exists and exports TIER1_DEFAULTS (frozen snapshot of ai/config.template.mjs) and re-exports AiConfig (live singleton).
• Per-server fixture files exist for each server-group with config.mjs imports in tests: memoryCoreDefaults.mjs, knowledgeBaseDefaults.mjs, githubWorkflowDefaults.mjs, neuralLinkDefaults.mjs.
• All assertion-style config.mjs imports in tests are migrated to the fixture-snapshot pattern.
• Mutation-style imports are NOT migrated (target same singleton; no drift risk).
• An operator-side smoke verifies: with a customized ai/config.mjs (e.g., modelProvider: 'openAiCompatible'), tests still pass.
• No regression in the 200+ existing affected specs.

Out of Scope

• Refactoring the operator-overlay mechanism itself.
• Touching the /tmp/*-config.mjs temporary fixtures (those are isolated test artifacts).
• Migrating the per-server templates' AiConfig import chain (anti-pattern per Option B rejection).

Sub-Decomposition

This is an umbrella; subdivide by server-group for review-ability:

• Sub-1: top-level ai/config.mjs (3 imports — smallest, validates the fixture pattern)
• Sub-2: knowledge-base (18 imports)
• Sub-3: github-workflow (8 imports)
• Sub-4: neural-link (4 imports)
• Sub-5: memory-core (46 imports — largest; ship last after pattern is settled)

Per-server subs reference this umbrella.

Avoided Traps

• ✓ Don't bulk-swap suffix only — doesn't close the second-order leak from per-server-template overlay chain.
• ✓ Don't rewire per-server templates back to top-level template — regresses #11969.
• ✓ Don't migrate mutation-style imports — they target the same Neo class singleton either way; no drift risk; churn without benefit.
• ✓ Don't add a "test-only" branch in the runtime config files — runtime-conditional behavior is an anti-pattern.

Related

• #11969 — corrective PR that exposed this concern (runtime now imports overlay; tests still do too)
• #11912 (potentially related — broader test-substrate cleanup epic)
• feedback_template_vs_overlay_import_v_b_a (memory authority for the V-B-A discipline that catches this class of bug)

Handoff Retrieval Hint: "test-config drift overlay vs template Option C fixture file 65 imports umbrella".