Resolves #15861

✅ AC-2 MET — two zero-retry samples at 229754695a. Ready for review.

sample	run	wall clock	result
1	31913052806 attempt 1	22:47:39Z → 22:53:39Z (6m00s)	13629 passed · 120 skipped · 0 flaky
2	31913052806 attempt 2	22:56:35Z → 23:02:43Z (6m08s)	13629 passed · 120 skipped · 0 flaky

Same head, no push between them — Playwright allocates fresh workers per attempt, which is exactly the resampling this AC asks for.

0 flaky is an inferred absence, so it carries a positive control. The line reporter omits the flaky line entirely at zero, which is indistinguishable from a reporter that never prints it. Run 31894997372 — the sample that found #17192 — emits 1 flaky in the identical format, so this instrument can see a present case and the absence is real.

Both AC-5 leaves are closed: #17186 (PR #17189) and #17192 (PR #17193, merged 22:45Z). No leaf is open, so per AC-6 this PR is approvable and this banner is where the merge gate reads that.

Nothing is stacked here any more — one commit ahead of dev, one file changed. The diff is the flip and nothing else.

Flips workers: process.env.CI ? 1 to 4 in the unit config, banking the ~2.7× wall-clock win measured on #15783.

Evidence: L4 (two zero-retry full-suite CI samples at one head, retry counts read from the runs' own output with a positive control; barrier and serializer verified from a JSON report's per-test worker assignment) → L4 required for every AC. Residual: none.

Deltas from ticket

The ticket is 22 days old and I drift-probed it before writing a line. It holds completely — worth stating, because two tickets I touched today had rotted premises:

One delta, and it is a weakness in AC-2 rather than in the code:

retries: 2 can satisfy AC-2 with a broken suite. The config retries twice in CI. An isolation defect that fails under parallelism and passes on retry reports as a pass — so "≥ 2 green full-suite CI samples" is satisfiable by exactly the failure mode this flip exists to surface, wearing a pass.

I have not changed retries. Moving two variables in one measurement would make a red unattributable — parallelism or retry-removal? Instead a sample counts only when its retry count is zero, which Playwright's JSON reporter records, so it is measurable without touching a second knob. The reasoning is recorded at the config line, not just here.

AC-2 is @neo-opus-ada's and I have asked her to rule rather than reinterpreting it myself. If she prefers retries: 0 for probe runs, that is a one-line follow-up.

What this PR is

A discovery instrument. Three isolation defects closed the first attempt (PR #15784, closed unmerged). Whether three were all of them is unknown — parallelism surfaces isolation defects probabilistically, which is precisely why the ticket demands two samples rather than one.

Per AC-5, a red re-land is a successful probe, not a failed ticket. If a fourth defect surfaces:

• revert the flip
• file it as its own enabler leaf, linked here
• do not weaken, skip, or loosen any spec to make the flip stick — the ticket's Out of Scope forbids it, and a spec that fails under parallelism has a real isolation defect

@tobiu flagged in advance that four workers will break several specs. That is an expected outcome of this instrument, not a surprise, and the disposition above is how it gets handled.

AC-4 verified before the flip

The barriers parallelism broke first are intact:

• unit-profiling — dependencies: ['unit'] (the barrier: it does not start until the bulk is over) plus workers: 1 (the cross-file serializer) plus fullyParallel: false
• the three Chroma-dependent projects keep dependencies: ['chroma-setup'], so a wide bulk cannot race the Chroma lifecycle

A project-level workers: 1 overriding a top-level workers: 4 is the mechanism #15790 established; this PR relies on it rather than re-deriving it.

The measurement AC-3 asks for, as observed times

Stated as run times rather than a multiplier, because the ticket asks for exactly that — and this comparison is unusually clean: dev at b59631e871 and this head are the same code plus the flip, run within twenty minutes of each other on the same runner class. One variable.

Both configurations ran 13629 tests. The suite grew by 82 while the enabler leaves were being fixed, so this is not the older 13547-test comparison rescaled — it is a fresh pair.

AC-4: the barrier and the serializer, verified at RUNTIME

Reading the config proves the fields are present, not that they hold under four workers — and the CI reporter cannot answer it either. Its log names no passing spec, so the absence of StoreFilterProfile in it is not evidence of anything; I checked that before trusting it, and the check is what sent me to a different instrument.

The instrument that can see it is the JSON reporter's per-test worker assignment, from a local wide run under CI=true:

bulk `unit` project      2735 tests   workerIndex {0, 1, 2, 3}   ← the flip is live
`unit-profiling`            2 tests   workerIndex {4}            ← project-level workers:1 wins

• Serializer holds. Both profiling specs ran on a single worker index, and zero pairs overlap in time. One worker cannot run two tests at once.
• Barrier holds. Last bulk test ended 23:06:57.447Z; first profiling test started 23:06:58.028Z — a 581ms gap and no overlap. I first compared start-to-start, which would have been satisfied by a barrier that did not hold; the end-to-start comparison is the one that tests the claim.

Test Evidence

Deliberately not claimed from local. The local runner cannot hold a wide unit run — the Chroma/web-server lifecycle contends — and single-worker local green says nothing about cross-file ordering that only exists at four. The ticket says so outright: "the falsifier only exists in CI at --workers=4." Reporting a local green here would be evidence for the wrong proposition.

The probe's first run — a defect, and the reason AC-2 changed

Run 31892182791 reported success with every check green — and carried 1 flaky with a Retry #1. That was the fourth isolation defect: MailboxService.spec.mjs:1381 used a fifty-turn budget as a stand-in for a condition, filed as #17186 and fixed by @neo-opus-ada in PR #17189.

Under AC-2 as originally written, that run would have counted as one of the two green samples.

Sample 1 — the first run that is actually evidence

Head ca6152c2ea (rebased onto #17189). Run 31894997372, attempt 1, unit job 95036852028, 16:15:24Z → 16:21:16Z.

Running 13638 tests using 4 workers
13547 passed (5.2m) · 120 skipped · 0 flaky · 0 retries

Zero-retry, and the zero is verified rather than assumed. Positive control: the identical grep found 1 flaky on the pre-fix run, so the pattern detects the thing whose absence is being claimed. Without that control, "no flaky line" and "my search was wrong" are the same output.

Evidence ledger

• Sample 1 — run 31894997372 attempt 1, flaky: 0, 13547 passed, 5.2m ✅
• Sample 2 — attempt 2, same head — flaky: 1, does not count. A fifth defect: HealthService.starvationFold.spec.mjs:265 asserts base.status === 'healthy' after probing the live plane, a precondition four workers can legitimately falsify. Filed as #17192.

It took BOTH rules composing, not either one

Sample 1 and sample 2 ran the same commit, the same config, the same runner class. Sample 1 was clean; sample 2 was not. Both reported CI success.

@neo-opus-ada's correction of my first framing here is worth carrying, because I had credited the two-sample rule alone and that is not what happened:

The two-sample rule produced a second observation, which is the only reason a differing result existed to see. The zero-retry rule made sample 2's 1 flaky disqualifying rather than a green anyone would have accepted. Either alone fails; the composition is what detects.

The two defects are also different mechanisms, which is the argument for continuing to sample rather than assuming a fixed inventory:

• Wall-clock — measured against a current baseline, see below

AC-3's "observed run times, not a claimed multiplier" earned itself twice

First against the arc's own ~2.7×, which was measured in July on a 9099-test suite. Today's suite is 13547 tests, so a July-baseline comparison measures suite growth and reports it as a parallelism regression.

Then against my own correction. I replaced it with a "current" baseline of 858s — the mean of two dev runs. @tobiu pointed out that earlier PRs had already been cutting CI time, which breaks the assumption that baseline is stationary. It is not:

08-14T18:02  980s ┐
08-14T21:27  975s ├─ before the perf work landed  ≈ 16m20s
08-15T07:59  981s ┘
08-15T09:02  905s ┐   #17128 merged 09:01 — "take the wake daemon spec
08-15T12:57  884s │    off CI's slow-file list"
08-15T15:14  855s ├─ after, drifting down as the fixed-sleep arc lands
08-15T16:14  827s ┘

Widening the sample from 2 to 10 made the number less accurate, because it pooled two regimes. More data, wrong answer — the population had a level shift in the middle of it.

The honest decomposition is two stacked wins, not one ratio:

The number this PR is entitled to claim is 2.47× — four workers against the current single-worker level, since that is what the flip actually replaces. The 2.80× belongs to the perf arc and this flip together, and #17128 is @neo-opus-ada's.

Disposition

Not merging on this evidence, and not reverting the branch either. AC-5 prescribes revert-and-file for a red re-land; this is greener than that and worse than it looks. The flip is correct and the defect it exposed is pre-existing, so reverting would re-hide #17186 rather than fix anything. The PR stays open and unmerged — which has the same effect as a revert for dev, without discarding the instrument.

The order is: #17186 lands, then two zero-retry samples here, then this merges.

Post-Merge Validation

AC-2 is unmet and this PR says so rather than implying otherwise. Merging on the current evidence would close #15861 on a sample that its own retry count disqualifies.

What is owed after #17186 lands: two full-suite CI samples at one head with flaky: 0, and the wall-clock delta restated against the #15783 baseline. The first observation — 6m08s at four workers — is recorded above but stays provisional, because a run containing a retry is not a clean timing sample either.

The revert clause AC-5 prescribes is deliberately not taken, and the Disposition section says why: the flip is correct and the defect it exposed is pre-existing, so reverting would re-hide #17186 instead of fixing it. Holding the PR unmerged protects dev identically without discarding the instrument. If @neo-opus-ada reads AC-5 as requiring the literal revert, it is one command and hers to call.

Evolution

The instructive part is what "green" is allowed to mean. This suite retries twice, and a retry is exactly the mechanism that converts a probabilistic isolation defect into a reported pass — so the metric the ticket chose to prove parallelism is safe is the same metric parallelism's failure mode can forge. Recording the retry count alongside the verdict costs nothing and is the difference between two samples that are evidence and two that are decoration.

Related: #15783 · #15789 · #15790 · #15847 Refs #15784 — the first attempt, closed unmerged because it worked as a probe.

Authored by Vega (Claude Opus 5, Claude Code). Session 5cd926fa-77e1-4309-8bbf-ca563ab07403.