LearnNewsExamplesServices
Frontmatter
id15783
titleMeasure unit-suite CI wall-clock with workers 4
stateClosed
labels
enhancementaitesting
assigneesneo-opus-vega
createdAtJul 24, 2026, 11:12 AM
updatedAtJul 24, 2026, 11:27 AM
githubUrlhttps://github.com/neomjs/neo/issues/15783
authorneo-opus-vega
commentsCount4
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJul 24, 2026, 11:27 AM

Measure unit-suite CI wall-clock with workers 4

Closed Backlog/active-chunk-9 enhancementaitesting
neo-opus-vega
neo-opus-vega commented on Jul 24, 2026, 11:12 AM

Context

Operator-named friction (2026-07-24 morning, continuing the 2026-07-23 discussion): the CI unit job takes ~10 minutes and is the longest pole on every non-docs PR. Evidence gathered in-session today: two recent successful runs measured 9:56 and 10:36 wall with setup (checkout + npm ci + parse5 bundle) only ~25s — the suite run IS the wall. The unit tree holds 820 spec files (560 under unit/ai). The decisive fact: test/playwright/playwright.config.unit.mjs:25 sets workers: process.env.CI ? 1 : undefined — the entire suite runs single-worker in CI on a 4-vCPU public runner.

git log -L archaeology: that line is unchanged since the config's creation (2025-10-12, #7471 granular-config split) — born-conservative, no recorded incident behind it. The isolation-by-construction substrate postdates it: per-process Chroma DB isolation (the #12335 orphan-incident fix), UNIT_TEST_MODE isolation by construction (ADR 0019 B4), and Playwright's per-file worker processes all landed after that default was chosen.

The Problem

Every code PR pays ~10 minutes of serial test execution whose parallelism ceiling was set once, two model-generations of substrate ago, and never revisited. Before any structural lever (sharding, suite splitting, changed-file selection — each with real cost or risk), the zero-cost lever is unmeasured: raise workers on the same runner. This ticket is the measurement, not the decision.

The Architectural Reality

  • test/playwright/playwright.config.unit.mjsfullyParallel: true, retries: process.env.CI ? 2 : 0, workers: process.env.CI ? 1 : undefined; projects unit (Body, pure Node) and unit-brain (/ai/ specs, Chroma via chroma-setup dependency).
  • Playwright workers are OS processes; cross-file singleton state does not collide. The plausible hazards under workers: 4 are: the SHARED Chroma instance behind unit-brain (mitigated by per-process DB naming from the #12335 fix), any spec binding a fixed port, and wall-clock-sensitive assertions on a loaded runner.
  • retries: 2 is already active in CI, so a latent cross-file collision surfaces as retry telemetry in the JSON report rather than a silent red.
  • GitHub-hosted ubuntu-latest for public repos: 4 vCPU — headroom for 4 workers exists.

The Fix (measurement protocol)

  1. Probe PR changing exactly one line: workers: process.env.CI ? 4 : undefined.
  2. Let the standard test.yml matrix run the FULL unit suite at the probe head (the config file is a non-docs path, so the classifier runs unit unconditionally); re-run the job for a second sample.
  3. Record wall-clock + retry/flake counts for ≥2 probe runs against the ≥2 baseline runs cited above (9:56, 10:36).
  4. Post the verdict on this ticket:
    • Wall-clock materially down AND no new flake class → hand the probe PR to the team + operator as an ordinary merge decision.
    • Any repeatable cross-file collision → close the probe PR unmerged and file the specific isolation defect it exposed (also a win — it names real, currently-masked coupling debt).

Acceptance Criteria

  • Probe PR diff is exactly the one workers line in playwright.config.unit.mjs.
  • ≥2 full unit CI runs at the probe head with wall-clock + retry counts recorded on this ticket next to the baseline numbers.
  • A verdict comment routes the outcome: merge-decision handoff (operator + team) OR probe closed with named isolation-defect ticket(s).
  • No silent merge: the probe PR merges only through the ordinary review + human merge gate after the verdict.

Out of Scope

  • Matrix sharding (doubles runner minutes), classifier suite-splitting (the unit/unit-brain project boundary already exists for a future 4th matrix row), and changed-file→spec selection (the risky lever — a prior targeted-selection miss on a pinning spec is documented evidence that selection needs transitive import-graph derivation; deliberately deferred).
  • Any change to the chroma setup/teardown projects or retries.

Avoided Traps

  • Do NOT treat a green probe as team-wide authority — the merge decision stays human; this ticket only converts opinion into data.
  • Do NOT tune workers and sharding simultaneously — one variable per measurement.

Related

  • #7471 (config birth, the unrevisited line) · #12335 (per-process Chroma isolation that makes parallelism plausible) · ADR 0019 B4 (UNIT_TEST_MODE isolation by construction) · #15368 (classifier suite-gating precedent for a future unit-brain split, out of scope here).

Decision Record impact: none.

Release classification: not release-blocking (CI throughput; boardless).

Live latest-open sweep: checked latest 20 open issues at 2026-07-24T09:13Z; no equivalent found. A2A claim sweep (all-status, last 12, ~50-min window): no overlapping [lane-claim]/[lane-intent] (recent claims #15780, #15782 are disjoint).

Origin Session ID: 2cca0fff-6354-4036-bfdd-fc3320938015

Retrieval Hint: query_raw_memories("unit CI single worker wall-clock workers 4 probe measurement")