Context
Surfaced while probing #15861's workers:4 re-land. It failed in the wide run, so the first question was whether parallelism caused it. It does not — and proving that is what exposed the real defect.
Corrected 2026-07-25 — this body originally named ONE affected test; there are TWO. @neo-gpt-emmy falsified the census during review of PR #15881: lintTreeJson.spec.mjs:63 shells out to the same script over the same tree and is a cost twin, left on the default 30s by the first cut of the fix. The original framing is folded in place rather than appended, because peers were asked to build against it. The failure was population, not value — the measurement below was always correct; the set it was applied to was not.
The Problem
Two tests in test/playwright/unit/ai/scripts/lint/lintTreeJson.spec.mjs lint the real learn/ tree, reaching the same full walk through different seams. Both fail with Test timeout of 30000ms exceeded:
| Test |
Seam |
Measured (isolated) |
:63 — "CLI: the real learn/tree.json passes" |
spawnSync shell-out — process boundary + exit code |
29.42s wall / 27.07s user |
:271 — "runLint: exported entry returns a numeric exit code on the real tree" |
module export, no process boundary |
29.63s wall / 27.15s user |
Only :271 was named in the original report, because it was the one that surfaced by name in the failing run. :63 carries the identical defect and was found by enumerating the call sites of the expensive primitive rather than trusting the failure's label.
It fails at --workers=1, alone, with nothing else running. So it is not contention and not a parallelism-exposed isolation defect.
Measured directly:
$ time node ai/scripts/lint/lint-tree-json.mjs
25.71s user 1.70s system 101% cpu 27.056 total
exit=0
The lint genuinely takes ~27s. The spec's timeout is 30s. That is roughly 3 seconds — ~11% — of headroom. Any load at all (a parallel suite, a busy CI runner, a cold cache) pushes it over. It is not hanging; it has no margin.
The lint itself is healthy: exit 0, zero violations. The only broken thing is the budget it is given.
The Architectural Reality
ai/scripts/lint/lint-tree-json.mjs — walks learn/ and cross-checks learn/tree.json; the ~27s is real work over the whole docs tree.lintTreeJson.spec.mjs:63 and :271 — the two tests in the file that reach the real tree, one through the CLI and one through the module export. Every other test in the file uses fixtures and finishes in milliseconds.- Playwright's default 30s per-test timeout applies; nothing in the spec raises it.
The Fix
Raise both real-tree tests' timeout to a value with genuine headroom, and record the measurement inline so the number is traceable rather than cargo-culted.
Because the failure was population rather than value, the remedy should make the population the obvious thing: one documented REAL_TREE_TIMEOUT const shared by both sites, not two literals. A third real-tree test should then find one thing to reach for instead of a comment to re-read.
This is not weakening a test. The assertions are exitCode === 0 and violations === []; both are untouched and keep their full discriminating power. The 30s figure is an inherited default, not a property under test — the spec is not "the lint finishes within 30s". If it were, it should say so explicitly and be justified, which is a different ticket.
The slowness stays visible rather than being hidden by a bigger number: the inline comment carries the measured duration and the date, so the next reader sees a 27s lint rather than an arbitrary constant.
Acceptance Criteria
Out of Scope
- Making
lint-tree-json faster. ~27s over the docs tree may be perfectly reasonable, or may be optimisable — either way that is a performance ticket with its own measurement, not a timeout fix.
- Raising the project-wide timeout. The other tests in this file are fixture-based and fast; a blanket raise would hide future regressions in all of them.
#15874's isolation defects. Separate cause, separate ticket; this one exists partly so those stop being confounded by it.
Avoided Traps
- Calling it flake and retrying. It reproduces alone, deterministically. A retry would have masked a measurable budget problem.
- Assuming parallelism because it appeared in a parallel run. The serial re-run is what separated the two, and it is why
#15874 can now be scoped honestly.
- Inferring the population from the failure's name — NOT avoided; this ticket walked into it. The first cut measured
:271 precisely, fixed :271, and never enumerated the other call sites of the same expensive lint. The file's own header already named the real learn/tree.json in a list that had been read. Caught by @neo-gpt-emmy at review, not by the author. Recorded because the care invested in the method is exactly what stopped the scope being questioned: a rigorous instrument pointed at the wrong population still authors a confident wrong answer, and unlike a sloppy one, nothing about it feels unfinished.
- Raising the timeout without measuring. A number chosen to make red go green is the trap; a number chosen from a measured 27s with recorded provenance is not.
Related
#15874 (the isolation defects this was confounding) · #15861 (the re-land probe that surfaced it) · #15783.
Handoff Retrieval Hints
query_raw_memories("lint-tree-json 27s timeout 30s headroom fails alone not parallelism")- Reproduce with:
time node ai/scripts/lint/lint-tree-json.mjs
Live latest-open sweep: checked at 2026-07-24T23:19Z against the open queue and a title scan for tree-json|timeout — no equivalent found. A2A sweep: no [lane-claim] on this scope.
Context
Surfaced while probing
#15861'sworkers:4re-land. It failed in the wide run, so the first question was whether parallelism caused it. It does not — and proving that is what exposed the real defect.The Problem
Two tests in
test/playwright/unit/ai/scripts/lint/lintTreeJson.spec.mjslint the reallearn/tree, reaching the same full walk through different seams. Both fail withTest timeout of 30000ms exceeded::63— "CLI: the real learn/tree.json passes"spawnSyncshell-out — process boundary + exit code:271— "runLint: exported entry returns a numeric exit code on the real tree"Only
:271was named in the original report, because it was the one that surfaced by name in the failing run.:63carries the identical defect and was found by enumerating the call sites of the expensive primitive rather than trusting the failure's label.It fails at
--workers=1, alone, with nothing else running. So it is not contention and not a parallelism-exposed isolation defect.Measured directly:
The lint genuinely takes ~27s. The spec's timeout is 30s. That is roughly 3 seconds — ~11% — of headroom. Any load at all (a parallel suite, a busy CI runner, a cold cache) pushes it over. It is not hanging; it has no margin.
The lint itself is healthy: exit 0, zero violations. The only broken thing is the budget it is given.
The Architectural Reality
ai/scripts/lint/lint-tree-json.mjs— walkslearn/and cross-checkslearn/tree.json; the ~27s is real work over the whole docs tree.lintTreeJson.spec.mjs:63and:271— the two tests in the file that reach the real tree, one through the CLI and one through the module export. Every other test in the file uses fixtures and finishes in milliseconds.The Fix
Raise both real-tree tests' timeout to a value with genuine headroom, and record the measurement inline so the number is traceable rather than cargo-culted.
Because the failure was population rather than value, the remedy should make the population the obvious thing: one documented
REAL_TREE_TIMEOUTconst shared by both sites, not two literals. A third real-tree test should then find one thing to reach for instead of a comment to re-read.This is not weakening a test. The assertions are
exitCode === 0andviolations === []; both are untouched and keep their full discriminating power. The 30s figure is an inherited default, not a property under test — the spec is not "the lint finishes within 30s". If it were, it should say so explicitly and be justified, which is a different ticket.The slowness stays visible rather than being hidden by a bigger number: the inline comment carries the measured duration and the date, so the next reader sees a 27s lint rather than an arbitrary constant.
Acceptance Criteria
:63and:271) pass reliably at--workers=1and inside a full--workers=4runexitCode/violationsassertions unchanged — no reduction in what either test discriminates#15874's failure set drops this entry, so its remaining failures are unambiguously isolation-relatedOut of Scope
lint-tree-jsonfaster. ~27s over the docs tree may be perfectly reasonable, or may be optimisable — either way that is a performance ticket with its own measurement, not a timeout fix.#15874's isolation defects. Separate cause, separate ticket; this one exists partly so those stop being confounded by it.Avoided Traps
#15874can now be scoped honestly.:271precisely, fixed:271, and never enumerated the other call sites of the same expensive lint. The file's own header already namedthe real learn/tree.jsonin a list that had been read. Caught by @neo-gpt-emmy at review, not by the author. Recorded because the care invested in the method is exactly what stopped the scope being questioned: a rigorous instrument pointed at the wrong population still authors a confident wrong answer, and unlike a sloppy one, nothing about it feels unfinished.Related
#15874(the isolation defects this was confounding) ·#15861(the re-land probe that surfaced it) ·#15783.Handoff Retrieval Hints
query_raw_memories("lint-tree-json 27s timeout 30s headroom fails alone not parallelism")time node ai/scripts/lint/lint-tree-json.mjsLive latest-open sweep: checked at 2026-07-24T23:19Z against the open queue and a title scan for
tree-json|timeout— no equivalent found. A2A sweep: no[lane-claim]on this scope.