Any package.json edit triggers the full browser components suite

Context

Friction surfaced 2026-08-01 by @tobiu watching CI on PR #16241: "triggering component ci when not touching any is pointless." The observation is correct, the cause is one line, and it is not the cause anyone would guess.

Filed with operator authorization. Everything below is observed from a live run; the prescription in The Fix is inference and is marked as such.

Live latest-open sweep: latest 20 open issues at 2026-08-01T01:30:24Z. A2A claim sweep over the last 12 messages at the same time: no [lane-claim] on CI scope or workflow routing (Iris holds #16233, Ada filed #16246 — neither overlaps).

The Problem

PR #16241 ran the full browser components suite for 2m34s on a diff containing zero component files — and then flaked in it, costing a re-run and a reviewer's time. The diff touched ai/, learn/, test/, and package.json. Not one file under src/, resources/scss/, or test/playwright/component/.

The scope classifier is not missing and is not broken. .github/workflows/test.yml has a changes job whose scope step emits run_unit / run_components / run_integration / run_parity, consumed by the suite matrix. Anyone approaching this expecting to build selection would be rebuilding something that already works. It selected correctly according to its own rule.

The rule is the problem, and it is one line:

const isComponentsRelevantPath = file =>
  file.startsWith('src/') ||
  file.startsWith('resources/scss/') ||
  file.startsWith('test/playwright/component/') ||
  file === 'test/playwright/configTemplateResolver.mjs' ||
  file === 'test/playwright/playwright.config.component.mjs' ||
  file === 'package.json' ||          // <- fires on ANY edit
  file === 'package-lock.json' ||
  file === '.github/workflows/test.yml';

package.json is one path carrying two unrelated kinds of change:

• dependencies / devDependencies — genuinely components-relevant. The suite mounts real components in a browser against the webpack dev server; a dependency shift can absolutely break that.
• scripts, version, metadata — cannot affect a browser build at all.

PR #16241's only package.json change was adding one npm script (ai:check-backup-integrity). That is what bought a 2m34s browser suite, a flake, and a re-run.

Why this is worth a ticket rather than a shrug: the cost is not only runner minutes. A suite that runs where it cannot be relevant can only produce false negatives — and it did exactly that here, turning a green PR red on FragmentDeltaUpdates, whose failing waitForSelector line is itself the fix from a previous flake ticket (#11538). Every such run is a chance to spend a review cycle on noise.

The Architectural Reality

• .github/workflows/test.yml — the changes job, scope step. Emits the four run_* booleans; the suite matrix consumes them at needs.changes.outputs.* and gates the steps with if: matrix.run == 'true'.
• Gating is at step level, not job level. Every suite job spawns regardless and appears in the check list; when its flag is false the test steps skip. So "the job appears" is expected and is not itself the defect — the defect is the flag being true.
• The coarseness is deliberate and documented, and any fix must respect why. In-file: "Keep this COMPLETE — a missed path silently skips the suite (#15368 — this suite ran in NO workflow at all until it was wired here)." The failure mode the current list defends against is a false negative, which is strictly worse than the false positive being reported here.
• .github/workflows/ is CI configuration, outside the ai/ tree — Agent OS Structure Map gate recorded N/A on that basis.

The Fix

(Prescription — inference, not observation.)

Split package.json by the kind of change, not by the path. A content-aware predicate: components are relevant when the diff of package.json touches dependencies / devDependencies (or when package-lock.json changes at all), and not when it touches only scripts / version / metadata.

Two properties any implementation must keep, because they are what makes the current coarse rule safe:

1. Fail toward running. If the change kind cannot be determined — parse failure, unexpected shape, missing base ref — treat it as relevant and run the suite. A false positive costs minutes; a false negative costs a shipped regression, which is exactly the #15368 history.
2. Same treatment for the other suites' package.json entries if they share the atom, so this does not become a components-only special case that drifts.

The same argument applies to .github/workflows/test.yml appearing in every whitelist — editing the classifier itself does not require running every suite — but that is deliberately not in scope here (see below): it is a second, weaker instance and bundling it would make the change harder to reason about.

Contract Ledger Matrix

Decision Record impact

none. No ADR governs CI suite selection.

Acceptance Criteria

• A PR whose only package.json change is under scripts / version / metadata resolves run_components=false.
• A PR touching dependencies or devDependencies still resolves run_components=true.
• Any package-lock.json change still resolves run_components=true.
• An undeterminable diff — parse failure, unexpected shape, missing base ref — resolves run_components=true, with a test that fails if the fallback is ever flipped to skip.
• The classifier's decision remains legible in the job log (it already prints run_components=…), so a future reader can confirm which predicate fired without re-deriving it.
• Post-merge: a real PR adding only an npm script shows the components steps skipped.

Out of Scope

• Unit-suite duration. ~13 minutes was raised in the same conversation, and it is a different problem: for a diff touching ai/, unit running is correct — ai/ is legitimately in its whitelist. That is suite size and parallelism, and it already has a home at #15861 (Re-land workers:4 in CI with two green full-suite samples, open, unassigned). Conflating the two would send this lane at selection when the answer there is sharding.
• .github/workflows/test.yml in every whitelist. Real, weaker, and separable (see The Fix).
• Job-level rather than step-level gating. Would remove the empty job entries from the check list, but changes required-check semantics — a distinct decision with its own blast radius.
• The FragmentDeltaUpdates flake itself. Surfaced by this run but independent; its waitForSelector hardening came from #11538.

Avoided Traps

• Narrowing the whitelist by deleting entries. The tempting version of this fix is "remove package.json". That reintroduces exactly the false-negative class #15368 was filed for — a dependency bump would then silently skip the browser suite. The fix must make the predicate smarter, never shorter.
• Assuming no selection exists. The 13-minute unit run and the pointless components run both look like "everything runs on everything". Selection is implemented and working; only one predicate is too coarse. Starting from the wrong premise here would produce a rewrite where a one-line refinement is warranted.
• Treating an appearing job as a running job. Steps are gated, not jobs, so a skipped suite still shows in the check list. Counting check entries is not a measure of what executed.

Related

• #15861 — unit-suite parallelism / workers:4 re-land; owns the duration half raised alongside this
• #15368 — wired the components suite into a workflow at all; the source of the completeness rule this must preserve
• #11538 — the FragmentDeltaUpdates viewport-wait hardening, the flake this run hit
• #16204 — devDependency classification; adjacent to package.json but about pruning, not CI routing
• PR #16241 — the run that surfaced it

Origin Session ID: 713db0da-2239-44ea-ba5b-931be90d34fc

Retrieval Hint: query_raw_memories("CI scope classifier package.json components suite whitelist"), or .github/workflows/test.yml isComponentsRelevantPath.