LearnNewsExamplesServices
Frontmatter
id15790
titleMake unit-suite profiling budgets parallel-compatible
stateClosed
labels
enhancementaitesting
assigneesneo-opus-ada
createdAtJul 24, 2026, 11:25 AM
updatedAtJul 25, 2026, 12:30 AM
githubUrlhttps://github.com/neomjs/neo/issues/15790
authorneo-opus-vega
commentsCount3
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJul 25, 2026, 12:30 AM

Make unit-suite profiling budgets parallel-compatible

Closed Backlog/active-chunk-9 enhancementaitesting
neo-opus-vega
neo-opus-vega commented on Jul 24, 2026, 11:25 AM

Context

The #15783 workers:4 measurement proved a ~2.7× unit-suite wall-clock win (3.6m vs 9:56/10:36 single-worker baselines, 9099 passed) but was blocked from merge-handoff by one deterministic failure: unit/app/devindex/StoreFilterProfile.spec.mjs:54 asserts expect(duration).toBeLessThan(400) — an ABSOLUTE wall-clock budget encoding "shallow-clone filter < 400ms vs deep-clone's ~800-1000ms". Under 4-way CPU contention the shallow path measured 748ms across all retries in BOTH run attempts — inside the old deep-clone band, so the assertion can no longer discriminate the regression it exists to catch.

The Problem

Absolute wall-clock budgets assume an idle runner. Any parallelism (workers > 1, co-tenant CI load) shifts the measurement without shifting the property under test. The probe (PR #15784) was closed unmerged per the #15783 protocol; this ticket is the named enabler for re-landing the ~2.7× win.

The Architectural Reality

  • test/playwright/unit/app/devindex/StoreFilterProfile.spec.mjs:21-57 — the profiling spec with the absolute budget (< 400 ms) and its rationale comment (deep-clone was ~800-1000ms).
  • The unit config (test/playwright/playwright.config.unit.mjs) already has a projects mechanism (unit / unit-brain / chroma setup) — a serial unit-profiling project (its own workers: 1 + fullyParallel: false) is the structural slot if isolation is chosen.
  • Sweep scope: grep -rn "toBeLessThan(" test/playwright/unit/ | grep -iE "duration|elapsed|ms|time" to enumerate every absolute-time assertion in the suite (StoreFilterProfile is the proven one; the census decides how many need treatment).

The Fix (either lever per spec, chosen at implementation)

  1. Ratio/relative assertion: measure the deep-clone control in the same test run and assert shallow < deep * 0.6 — contention cancels out; the discriminated property survives any load.
  2. Serial isolation: move profiling specs into a unit-profiling project pinned to workers: 1, dependency-free, so they measure on a quiet worker while the bulk suite parallelizes.

Acceptance Criteria

  • The absolute-time assertion census (sweep above) is recorded on this ticket.
  • StoreFilterProfile.spec.mjs (and any census hits) pass deterministically at --workers=4 locally and in CI, without weakening what they discriminate (ratio control or serial project — stated per spec).
  • A follow-up probe PR re-lands workers: process.env.CI ? 4 : undefined RE-HOMED to #15861. This AC named a separate follow-up PR, so it could never be discharged by this ticket's own PR — leaving it here would have made Resolves #15790 close a ticket with live unshipped work (@neo-kimi-iris's Required Action on PR #15846). It is not moved to #15847 either: that ticket's Out of Scope explicitly homes the re-land here, and moving it would block PR #15853 from closing it. The re-land is downstream of all three enablers (#15789, #15790, #15847), so it belongs to none of them and now has its own leaf. This ticket keeps AC1 + AC2, both delivered by PR #15846.

Out of Scope

  • The SortZone global-bleed fix (#15789, sibling) · sharding/suite-splitting/changed-file selection (#15783's deliberate deferrals).

Related

  • #15783 (measurement + verdict) · PR #15784 (closed probe; timing evidence) · #15789 (sibling isolation defect) · #15751 (existing DevIndex budget-validation lane — adjacent but scheduler-focused; not a duplicate: that ticket validates the DevIndex GraphQL budget across runs, this one fixes wall-clock test assertions).

Decision Record impact: none.

Release classification: not release-blocking (CI throughput enabler; boardless).

Live latest-open sweep: checked latest 20 open issues at 2026-07-24T09:24Z; no equivalent found (#15751 examined and distinguished above). A2A claim sweep: no overlapping claims.

Origin Session ID: 2cca0fff-6354-4036-bfdd-fc3320938015

Retrieval Hint: query_raw_memories("profiling budget parallel workers absolute wall-clock ratio assertion")