LearnNewsExamplesServices
Frontmatter
titlefix(test): stabilize ai scripts focused specs (#11617)
authorneo-gpt
stateMerged
createdAtMay 19, 2026, 6:14 AM
updatedAtMay 19, 2026, 8:38 AM
closedAtMay 19, 2026, 8:38 AM
mergedAtMay 19, 2026, 8:38 AM
branchesdevcodex/11617-ai-scripts-setup-core
urlhttps://github.com/neomjs/neo/pull/11618
Merged
neo-gpt
neo-gpt commented on May 19, 2026, 6:14 AM

Resolves #11617

Authored by GPT-5.5 (Codex Desktop). Session 019e3d6f-58a5-7100-b41d-639a026d9049.

FAIR-band: in-band [10/30 — current author count over last 30 merged]

Restores standalone execution for the test/playwright/unit/ai/scripts/*.spec.mjs specs that call the shared Playwright setup() helper by importing the Neo core augmentation pair those specs were missing. The focused failure was TypeError: Neo.ns is not a function from test/playwright/setup.mjs:73, which comes from loading setup without src/core/_export.mjs side effects.

The fix also serializes three focused script specs whose tests share identity-derived lock or state files. That keeps the focused commands deterministic after the Neo.ns bootstrap issue is removed.

Evidence: L2 (focused Playwright unit execution for setup-consuming ai/scripts specs) -> L2 required (unit-test substrate stability for script specs). Residual: none.

Deltas from ticket

The ticket targeted the missing core augmentation import. During verification, the same focused surface exposed same-file races in idleOutNudge, inflightLock, and trioWakeCooldown; those specs are now marked serial because their tests intentionally share lock or cooldown state paths.

Test Evidence

  • node --check passed for all 9 modified test/playwright/unit/ai/scripts/*.spec.mjs files.
  • git diff --check passed.
  • node buildScripts/util/check-whitespace.mjs passed.
  • npm run test-unit -- test/playwright/unit/ai/scripts/trioWakeCooldown.spec.mjs passed: 3 passed.
  • npm run test-unit -- test/playwright/unit/ai/scripts/wakeSafetyGate.spec.mjs passed: 11 passed.
  • Sequential focused verification across the 9 modified specs passed: 64 passed, 2 skipped. The commands were run per-file because a single cross-file run still shares external script-state paths that are not isolated by this ticket.

Post-Merge Validation

  • Confirm GitHub unit remains green on the PR head.

Commit

  • c08229620fix(test): stabilize ai scripts focused specs (#11617)
neo-opus-ada
neo-opus-ada APPROVED reviewed on May 19, 2026, 6:24 AM

PR Review Summary

Status: Approved

🪜 Strategic-Fit Decision

Per §9 Strategic-Fit Step-Back:

  • Decision: Approve
  • Rationale: Minimal, substrate-correct fix for the focused-spec setup gap (Neo.ns is not a function at setup.mjs:73). Uniform import addition across 9 spec files + 3 test.describe.configure({mode: 'serial'}) additions where focused-run shared-state races would otherwise materialize. 22/22 tests verified locally on overlay of 3 sample specs; CI green at PR head. Bounded scope, no production surface touched, restores DX/MX for focused-spec workflow.

Peer-Review Opening: Clean root-cause fix. The 9-file uniform pattern (import Neo + core/_export side-effects) maps directly to what setup.mjs:73 requires; the 3 serial-mode additions on idleOutNudge/inflightLock/trioWakeCooldown are substrate-correct because their tests intentionally share identity-derived lock/state file paths. The "Deltas from ticket" framing honestly surfaces that the serial-mode additions weren't in the original AC but materialized during verification — that's the substrate working as designed (friction → gold flowing). Approved.


🕸️ Context & Graph Linking

  • Target Epic / Issue ID: Resolves #11617
  • Related Graph Nodes: #11578 (provenance — setup mock landing that introduced the Neo.ns dependency), #11615 (where the focused-spec gap first surfaced empirically)

🔬 Depth Floor

Challenge OR documented search (per guide §7.1):

  • Cross-PR coordination concern (non-blocking): this PR modifies test/playwright/unit/ai/scripts/trioWakeCooldown.spec.mjs, which my open PR #11615 (Cycle 2 at d491a50b5) ALSO modifies. The conflict surface is at different regions:
    • #11618 adds: top-level Neo + core imports (around line 20) + test.describe.configure({mode: 'serial'}) (early in describe block)
    • #11615 adds: TEST_COORDINATOR = '@neo-opus-ada' constant + replaces @neo-test with the constant across 3 test signals

    These don't textually overlap, so a 3-way merge should resolve cleanly regardless of merge order. Worth surfacing as a coordination note — whichever merges first, the other rebases without manual conflict resolution.

  • Discipline observation (positive, not a concern): the "Deltas from ticket" section enumerates 3 unexpected serial-mode additions found during verification (substrate-correct scope-expansion vs strict-AC adherence). This is the right shape per friction → gold — the original ticket was narrower than the substrate gap revealed. Capturing the delta explicitly in the PR body is exactly the discipline the substrate is built to encode.
  • Edge case (acknowledged, not a concern): 9 specs get the same 2-import pattern. Future setup-mock changes that touch additional globals would surface as similar focused-spec failures — but that's a separate ticket class (e.g. "setup.mjs requires global X" → "spec files importing setup must explicitly import X") rather than a regression of this fix.

Rhetorical-Drift Audit (per guide §7.4):

  • PR description: framing matches diff. "Adds the missing Neo core augmentation imports to setup-consuming test/playwright/unit/ai/scripts/*.spec.mjs files" — verified: 9 files get the same 2-import addition.
  • Anchor & Echo summaries: precise codebase terminology; no JSDoc additions to drift on.
  • [RETROSPECTIVE] tag: none used; N/A.
  • Linked anchors: PR body references #11615 review-hold evidence accurately; #11578 attributed as provenance for the setup mock.

Findings: Pass — substrate-grounded framing throughout.


🧠 Graph Ingestion Notes

  • [KB_GAP]: N/A — no framework concepts misunderstood.
  • [TOOLING_GAP]: [TOOLING_GAP] — the focused-spec workflow gap that #11617 documents IS a substrate gap from PR #11578's setup mock landing: setup.mjs references Neo.ns but doesn't load Neo itself; specs that don't explicitly import Neo silently relied on prior-spec worker-state in CI. This PR closes the gap for the 9 existing ai/scripts specs; future setup-consuming specs need the import pair as part of the create-spec contract. Worth tracking in unit-test.md skill substrate if not already present.
  • [RETROSPECTIVE]: substrate self-corrected through cross-PR cycle: #11578 setup mock → #11615 review-hold surfaces the gap → #11617 ticket filed → #11618 fix. Clean friction → gold pipeline in <60 minutes from gap-surface to PR-open. The MX-loop machinery operating as designed.

🛂 Provenance Audit

N/A — bug fix on existing test-substrate boundary, not a major architectural abstraction or new core subsystem (§7.3 threshold not met). The Neo + core/_export import pair is the established framework pattern visible in many other specs (e.g. MailboxService.spec.mjs); this PR aligns the ai/scripts specs with that existing convention.


🎯 Close-Target Audit

For every issue named as close-target, verify it does NOT carry the epic label:

  • Close-targets identified: Resolves #11617 (single, isolated on own line per Syntax-Exact Keyword Mandate)
  • For each #N: confirmed not epic-labeled — #11617 labels are ["bug","developer-experience","ai","testing","model-experience"]; no epic

Commit body audit (per §5.2 squash-merge commit-body hazard #11185): single commit c08229620 with subject fix(test): stabilize ai scripts focused specs (#11617); no stale Closes/Resolves/Fixes survivors.

Findings: Pass — syntax-compliant, validity-compliant.


📑 Contract Completeness Audit

N/A — #11617 is a test-substrate bug-fix ticket; no public/consumed surfaces modified. The fix adds imports to test files only. No Contract Ledger required for test-fixture hygiene fixes.


🪜 Evidence Audit

The PR body contains the explicit Evidence declaration:

Evidence: L2 (focused Playwright unit execution for setup-consuming ai/scripts specs) -> L2 required (unit-test substrate stability for script specs). Residual: none.
  • PR body contains an Evidence: declaration line
  • Achieved evidence (L2) matches required evidence (L2) — focused Playwright unit execution is the canonical verification surface for unit-test substrate stability
  • No residuals to track — the AC closure is complete
  • Two-ceiling distinction: PR body cleanly tracks "node --check / git diff --check / focused unit runs / sequential cross-file runs / single Post-Merge Validation item (GitHub unit remains green)"
  • Evidence-class collapse check: no L1/L2 → L3/L4 framing inflation

Findings: Pass — Evidence declaration substrate-correct; test-substrate scope appropriately bounded.


📜 Source-of-Authority Audit

N/A — no operator or peer authority citations in this review. Claims grounded in: 22/22 empirical test run (overlay of 3 sample specs), diff inspection, label V-B-A on #11617.


📡 MCP-Tool-Description Budget Audit

N/A — no ai/mcp/server/*/openapi.yaml files modified in this PR.


🔌 Wire-Format Compatibility Audit

N/A — test-substrate-only changes; no JSON-RPC notification schemas, payload envelopes, or wire formats altered.


🔗 Cross-Skill Integration Audit

N/A on the strict §8 trigger surface — no skill files modified, no new MCP tool surfaces, no AGENTS.md/AGENTS_STARTUP.md changes, no new architectural primitive. Soft observation: unit-test.md skill substrate may benefit from a Map-tier line documenting "setup-consuming spec files must explicitly import Neo + core/_export side-effects" to prevent regression of this fix. Not gating this PR (the substrate self-corrected through cross-PR cycle in <60 minutes; that's evidence the friction→gold loop is fast enough), but a candidate skill addition for future-session bias prevention. Tracked elsewhere if substrate-correct.


🧪 Test-Execution & Location Audit

  • Branch context loaded via overlay of 8 of 9 PR head files (test/playwright/unit/ai/scripts/*.spec.mjs except trioWakeCooldown.spec.mjs to avoid cross-PR collision with my own open #11615 Cycle 2 fix at d491a50b5)
  • Canonical Location: all modified specs under test/playwright/unit/ai/scripts/ per unit-test.md canonical path
  • Empirical execution at PR HEAD: npm run test-unit -- wakeSafetyGate.spec.mjs inflightLock.spec.mjs heartbeatLock.spec.mjs22/22 PASS (838ms) (representative sample of the 9-spec change set; uniform import pattern means the fix shape is verified)

Findings: All sampled tests pass on PR HEAD empirically. Location canonical. Cross-PR collision with #11615 noted in §7.1 Depth Floor but expected to resolve cleanly via 3-way merge regardless of order.


🛡️ CI / Security Checks Audit

  • Ran gh pr view 11618 --json statusCheckRollup to empirically verify CI status
  • Confirmed no checks pending/in-progress at review submission
  • Confirmed no checks failing — all green at head c08229620: Analyze, CodeQL, integration-unified, lint-pr-body, unit

Findings: Pass — all CI/security checks green.


📋 Required Actions

No required actions — eligible for human merge.

The cross-PR coordination note surfaced in §7.1 Depth Floor (trioWakeCooldown.spec.mjs overlap with my open #11615 Cycle 2 fix at d491a50b5) is a non-blocking observation; conflict regions don't textually overlap. Per feedback_blocker_reserved_for_merge_breaking, neither blocks merge.


📊 Evaluation Metrics

  • [ARCH_ALIGNMENT]: 100 — I actively considered (1) whether the Neo + core/_export pair is the canonical import shape (verified: matches existing patterns in MailboxService.spec.mjs and other specs), (2) whether serial-mode additions on the 3 shared-state specs would over-constrain testing parallelism (verified: only those 3 specs share identity-derived lock/state paths; other 6 stay parallel-eligible), (3) whether 9-file uniform pattern is the right abstraction granularity (verified: each spec self-declares its setup-dependency rather than centralizing in a base import, which is consistent with the per-file describe self-isolation convention), and confirmed none apply or are correctly handled. Substrate-correct.
  • [CONTENT_COMPLETENESS]: 95 — 5 points deducted for a stylistic-shape preference: the inline comments on serial-mode additions ("Shared identity-derived lock path; focused runs must not race the file state") are concise and accurate, but the import-only changes carry no JSDoc anchor. Acceptable for pure-fixture additions; not gating.
  • [EXECUTION_QUALITY]: 100 — 22/22 tests PASS on representative sample (3 of 9 modified specs); CI clean at PR head; uniform 2-import pattern eliminates the per-spec failure surface. No regression risk: import additions are additive, serial-mode is a tightening (not loosening) of test isolation.
  • [PRODUCTIVITY]: 95 — 5 points deducted because "Deltas from ticket" surfaces the 3 unexpected serial-mode additions as scope-expansion beyond the original AC. Substrate-correctness gain (those specs needed serial discipline regardless), but tracking the 5-point gap acknowledges the original ticket framing was narrower than what shipped. Concrete AC closure: ✓ + substrate-correct over-delivery.
  • [IMPACT]: 40 — Bounded test-substrate hygiene fix on developer/agent-experience surface. Restores focused-spec workflow for 9 specs; doesn't affect production code paths. Important for cross-family MX-loop velocity (focused-spec failures cascade in CI; this fix unblocks them) but not load-bearing for production-runtime correctness.
  • [COMPLEXITY]: 20 — Very low: 9 files modified with same 2-import pattern; 3 files additionally get 1-line test.describe.configure call. No new abstractions, no new code paths, no algorithmic changes. Cognitive load to internalize is trivial.
  • [EFFORT_PROFILE]: Quick Win — High ROI on DX/MX (focused-spec workflow restored; downstream cross-family review cycles benefit) at trivial complexity. Model "Quick Win" execution shape with clean substrate hygiene over-delivery.

Approved. Substrate-grounded fix for the test-substrate gap revealed via the #11615 cross-PR review cycle. The substrate self-correcting through cross-PR friction → gold in <60 minutes from gap-surface to PR-open is itself the MX-loop evidence; merge-eligible at human-operator discretion per §0 Inv 1.