LearnNewsExamplesServices
Frontmatter
id12682
titleQueryRecentTurns.spec leaks an unrestored embedText mock into sibling specs
stateClosed
labels
bugaitesting
assigneesneo-opus-grace
createdAtJun 7, 2026, 12:47 PM
updatedAtJun 7, 2026, 4:57 PM
githubUrlhttps://github.com/neomjs/neo/issues/12682
authorneo-opus-grace
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 7, 2026, 4:57 PM

QueryRecentTurns.spec leaks an unrestored embedText mock into sibling specs

neo-opus-grace
neo-opus-grace commented on Jun 7, 2026, 12:47 PM

Context

QueryRecentTurns.spec.mjs (added by #12671) patches TextEmbeddingService.embedText in its beforeAll to return an offline 4096-length vector, but its afterAll restores only StorageRouter.getMemoryCollectionnot embedText. The mock therefore leaks into any sibling spec that shares the same Playwright worker. Surfaced 2026-06-07 when it reddened the unit CI lane on the docs-only PR #12681 (#9994).

Diagnosed by @neo-gpt (read-only V-B-A: ordered repro + root-cause + precedent).

The Problem

Playwright runs specs in shared workers (fullyParallel). When QueryRecentTurns.spec.mjs runs before TextEmbeddingService.retry.spec.mjs in the same worker, the unrestored embedText override makes TextEmbeddingService.retry.spec.mjs:216 receive the 4096-length 0.1 vector instead of its expected [0.1, 0.2, 0.3] → red. Because worker assignment/ordering is non-deterministic, this is a latent flake that reddens unrelated PRs (here, the docs-only #12681).

Ordered repro (fails like CI on the unfixed spec):

npm run test-unit -- test/playwright/unit/ai/services/memory-core/QueryRecentTurns.spec.mjs test/playwright/unit/ai/services/memory-core/TextEmbeddingService.retry.spec.mjs --workers=1

The Architectural Reality

  • Leak site: test/playwright/unit/ai/services/memory-core/QueryRecentTurns.spec.mjsbeforeAll sets TextEmbeddingService.embedText = async () => new Array(4096).fill(0.1); afterAll restores getMemoryCollection but not embedText.
  • Victim: test/playwright/unit/ai/services/memory-core/TextEmbeddingService.retry.spec.mjs:216.
  • Precedent (correct pattern): test/playwright/unit/ai/services/memory-core/HealthService.spec.mjs saves originalEmbedText before patching + restores it in teardown.
  • Class: method-mock-on-singleton not restored — sibling of #12435's aiConfig-singleton-mutation, but a distinct object (TextEmbeddingService.embedText, not aiConfig), so it is its own narrow fix rather than part of #12435.

The Fix

Mirror the file's existing originalGetMemoryCollection save/restore for embedText:

  • declare originalEmbedText in the describe-scope let,
  • originalEmbedText = TextEmbeddingService.embedText before the override,
  • if (originalEmbedText) TextEmbeddingService.embedText = originalEmbedText in afterAll.

Acceptance Criteria

  • AC1: QueryRecentTurns.spec.mjs restores TextEmbeddingService.embedText in afterAll.
  • AC2: the ordered repro above passes (no cross-spec leak).
  • AC3: QueryRecentTurns.spec.mjs + TextEmbeddingService.retry.spec.mjs remain green standalone.

Out of Scope

  • The aiConfig-singleton-mutation isolation (#12435) — distinct object/scope.
  • A broader audit of every spec's mock-restore hygiene — this ticket fixes the one confirmed leak blocking #12681; a systemic sweep is a separate concern if warranted.

Related

  • #12671 — origin of QueryRecentTurns.spec.mjs (CLOSED).
  • #12681 / #9994 — the docs PR reddened by this leak (unblocked once this merges).
  • #12435 — sibling test-isolation class (aiConfig singleton; distinct object).

Decision Record impact: none (test-isolation hygiene; aligned with the restore/by-construction isolation discipline, no ADR amendment).

Origin Session ID: 2d558ccb-e067-4777-bb80-e52f86d5ca43 Retrieval Hint: "embedText mock leak QueryRecentTurns sibling spec CI red" Live latest-open sweep: checked latest 20 open issues 2026-06-07; no equivalent found.

tobiu referenced in commit fed2944 - "fix(ai): restore embedText mock in QueryRecentTurns.spec afterAll (#12682) (#12683)" on Jun 7, 2026, 4:57 PM
tobiu closed this issue on Jun 7, 2026, 4:57 PM