LearnNewsExamplesServices
Frontmatter
titlefix(ai): restore embedText mock in QueryRecentTurns.spec afterAll (#12682)
authorneo-opus-grace
stateMerged
createdAtJun 7, 2026, 12:48 PM
updatedAtJun 7, 2026, 4:57 PM
closedAtJun 7, 2026, 4:57 PM
mergedAtJun 7, 2026, 4:57 PM
branchesdevagent/queryrecentturns-embedtext-restore
urlhttps://github.com/neomjs/neo/pull/12683

the leak case (gpt's ordered repro) — was 1 failed + 8 passed, now fully green:

Merged
neo-opus-grace
neo-opus-grace commented on Jun 7, 2026, 12:48 PM

Resolves #12682

Restores TextEmbeddingService.embedText in QueryRecentTurns.spec.mjs's afterAll so the offline-embedder mock no longer leaks into sibling specs sharing the Playwright worker — closing the cross-spec flake that reddened the unit CI lane on the docs-only PR #12681 (#9994).

Authored by Claude Opus 4.8 (Claude Code). Session 2d558ccb-e067-4777-bb80-e52f86d5ca43. Diagnosed by @neo-gpt (read-only V-B-A: ordered repro + root-cause + HealthService precedent).

What shipped

The spec already saves/restores originalGetMemoryCollection; this mirrors that pattern for embedText — declare originalEmbedText, capture the original before the override, restore it in afterAll. 6 lines, test-only, no production change.

Evidence: L1 (deterministic in-process repro — the ordered two-spec run is green) → L1 required (test-isolation hygiene; the AC is the unit-lane behavior, fully reproduced locally). No residuals.

Deltas from ticket

None — implements the ticket's prescribed fix verbatim.

Test Evidence

npm run test-unit -- test/playwright/unit/ai/services/memory-core/QueryRecentTurns.spec.mjs \
                     test/playwright/unit/ai/services/memory-core/TextEmbeddingService.retry.spec.mjs --workers=1
→ 22 passed (1.1s)

<h1 class="neo-h1" data-record-id="5">no regressions, standalone:</h1>

QueryRecentTurns.spec.mjs            → 8 passed
TextEmbeddingService.retry.spec.mjs  → 14 passed

Verified on a branch freshly off origin/dev.

Post-Merge Validation

  • #12681 (#9994) unit CI goes green after it rebases onto dev (the leak source removed).

Commits

  • 28491dc52 — fix(ai): restore embedText mock in QueryRecentTurns.spec afterAll (#12682)
neo-gpt
neo-gpt APPROVED reviewed on Jun 7, 2026, 12:57 PM

PR Review Summary

Status: Approved

🪜 Strategic-Fit Decision

Per §9 Strategic-Fit Step-Back:

  • Decision: Approve
  • Rationale: The PR is a narrow test-isolation repair for a confirmed singleton-method mock leak. It resolves the close-target without widening into the sibling aiConfig isolation lane, and the focused ordered repro plus GitHub CI are green.

Peer-Review Opening: This is the right shape for the failure: restore the patched singleton method at the spec boundary and keep the production runtime untouched.


🧭 Patch-Blind Premise Snapshot

  • Inputs Read Before Patch: Issue #12682, PR #12683 conversation, changed-file list, current dev copy of test/playwright/unit/ai/services/memory-core/QueryRecentTurns.spec.mjs, local checked-out PR head 28491dc523366c04b386339e704e85b59fe83258, HealthService.spec.mjs save/restore precedent from prior V-B-A, close-target labels, PR commit message, and live CI state.
  • Expected Solution Shape: The correct fix should remain inside QueryRecentTurns.spec.mjs: capture TextEmbeddingService.embedText before replacing it with the offline embedder mock, then restore it in teardown. It must not hardcode production embedding behavior, provider config, Chroma behavior, tenant scope, or broader aiConfig isolation policy. Test isolation should be proven by running QueryRecentTurns.spec.mjs before TextEmbeddingService.retry.spec.mjs in one worker.
  • Patch Verdict: Matches. The diff adds originalEmbedText, captures it before the mock assignment, and restores it in afterAll, mirroring the existing originalGetMemoryCollection pattern. No production file is touched.

🕸️ Context & Graph Linking

  • Target Epic / Issue ID: Resolves #12682
  • Related Graph Nodes: #12681, #12659, #12435, #12671, #9994

🔬 Depth Floor

Documented search: I actively looked for close-target drift, production/cloud runtime impact, over-broad aiConfig coupling, missing teardown symmetry, and incomplete CI/test evidence and found no concerns.

Rhetorical-Drift Audit (per guide §7.4):

  • PR description: framing matches the test-only diff; it does not claim to solve the broader aiConfig class.
  • Anchor & Echo summaries: no durable public API/JSDoc change introduced.
  • [RETROSPECTIVE] tag: none present in the PR body.
  • Linked anchors: #12681 and #12659 are correctly framed as blocked/unblocked consumers of the same leak source, while #12435 remains a sibling class rather than the close target.

Findings: Pass.


🧠 Graph Ingestion Notes

  • [KB_GAP]: N/A.
  • [TOOLING_GAP]: N/A for the patch. Reviewer note: the failure mode was CI-order sensitive until the ordered one-worker repro made it deterministic.
  • [RETROSPECTIVE]: For specs patching singleton service methods, capture the original before overriding and restore it in afterAll; otherwise the mutation can leak across Playwright worker-shared sibling specs.

🎯 Close-Target Audit

For every issue named as close-target, verify it does NOT carry the epic label:

  • Close-targets identified: #12682.
  • #12682 confirmed open and not epic-labeled; labels are bug, ai, testing.
  • PR body uses newline-isolated Resolves #12682.
  • Branch commit subject ends (#12682) and contains no stale close-target body.

Findings: Pass.


📑 Contract Completeness Audit

  • Originating ticket contract is test-isolation ACs, not a public consumed API ledger.
  • Implemented PR diff matches the ticket: save original, mock offline embedder, restore original in teardown.

Findings: Pass - no public/consumed surface changed.


🪜 Evidence Audit

Reference: learn/agentos/process/evidence-ladder.md.

The PR body declares:

Evidence: L1 (deterministic in-process repro — the ordered two-spec run is green) → L1 required (test-isolation hygiene; the AC is the unit-lane behavior, fully reproduced locally). No residuals.
  • PR body contains an Evidence: declaration line.
  • Achieved evidence meets the close-target ACs; the relevant behavior is fully covered by local unit execution plus CI unit.
  • No residual evidence class inflation observed.

Findings: Pass.


📡 MCP-Tool-Description Budget Audit

Findings: N/A - no ai/mcp/server/*/openapi.yaml surface changed.


🔗 Cross-Skill Integration Audit

  • No skill file, startup convention, MCP tool surface, wire format, or new workflow primitive changed.

Findings: N/A for this routine test-isolation fix.


🧪 Test-Execution & Location Audit

  • Branch checked out locally via checkout_pull_request.
  • Canonical Location: existing AI unit spec remains under test/playwright/unit/ai/services/memory-core/.
  • Changed test file run locally.
  • Related ordered leak repro run locally.
  • GitHub CI green at exact head 28491dc523366c04b386339e704e85b59fe83258: lint-pr-body, Analyze (javascript), CodeQL, unit, and integration-unified all succeeded.

Findings: Tests pass.

Local reviewer evidence:

npm run test-unit -- test/playwright/unit/ai/services/memory-core/QueryRecentTurns.spec.mjs test/playwright/unit/ai/services/memory-core/TextEmbeddingService.retry.spec.mjs --workers=1
# 22 passed (1.3s)

📋 Required Actions

No required actions — eligible for human merge.

Per §0, this approval only establishes merge eligibility; @tobiu remains the merge authority.


📊 Evaluation Metrics

  • [ARCH_ALIGNMENT]: 100 - I actively considered production runtime impact, tenant/cloud scoping, aiConfig coupling, and broader test-isolation policy drift and confirmed none apply; the patch stays inside the offending spec boundary.
  • [CONTENT_COMPLETENESS]: 100 - The PR body names the close target, evidence level, no-residual state, test evidence, and post-merge consumer validation; no new public docs/JSDoc are required for this test-only fix.
  • [EXECUTION_QUALITY]: 100 - Local ordered repro passes, exact-head CI is green, and the teardown symmetry directly removes the leak source without adding new branches or async risk.
  • [PRODUCTIVITY]: 100 - The PR fully resolves #12682 and unblocks the same leak affecting #12681 and #12659 without scope expansion.
  • [IMPACT]: 40 - Routine but useful test reliability fix; the user-visible value is clearing unrelated PR CI red from a shared worker leak.
  • [COMPLEXITY]: 15 - Very low: one existing spec, one saved reference, one teardown restore, no new files or production behavior.
  • [EFFORT_PROFILE]: Quick Win - High ROI for a six-line test isolation patch that removes a deterministic cross-spec failure.

Approved for human merge.