LearnNewsExamplesConnect
Frontmatter
assigneesneo-opus-grace
createdAtNov 5, 2026, 3:24 PM
updatedAtNov 5, 2026, 6:17 PM
githubUrlhttps://github.com/neomjs/neo/issues/20329
authorneo-opus-grace
commentsCount1
parentIssue20328
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtNov 5, 2026, 6:17 PM

The scope classifier admits e2e tiers for paths they cannot reach

Closed v13.4.0/archive-v13-4-0-chunk-3 enhancementaiperformancebuild
neo-opus-grace
neo-opus-grace commented on Nov 5, 2026, 3:24 PM

Context

neo PR #20321 changed learn/blog/persistent-memory-for-ai-coding-agents.md and apps/portal/resources/data/blog.json. It ran the engine e2e tier (4 shards) and the Brain-gated tier (10 shards) in full, and its Brain shards queued 7–20 minutes on the organization's 20-job cap. The operator, 2026-11-05: "you changed a markdown blog post file, and that triggered ALL unit and E2E pipelines to run. this is horribly wasteful." Baseline and method: #20328.

The Problem

classify-test-scope.yml admits a tier whenever a changed path falls inside its boundary. Four admissions reach nothing the tier runs. Counts are over the 346 PRs merged into dev from 2026-10-13 to 2026-11-05:

Admitted by Why the tier cannot reach it PRs
apps/portal/resources/data/**, through apps/ No e2e spec, fixture or config reads portal data; unit's content guards own it 6 (every blog post)
buildScripts/{create,docs,helpers,release}/**, through buildScripts/ Outside the e2e launch chain (below) 8
A neo-agent-skills-only bump (package.json + package-lock.json) Agent instructions, which webpack and Playwright never load. Unit's prepare.spec.mjs and baseline / Skills materialized cover its install path 12
Any change outside the E2E boundary, docs included The Brain tier still runs a one-shard smoke. It tests the Brain's dev tip, not the change, and the dev push runs the full tier anyway 67 today, 93 after the rows above

The Architectural Reality

  • isE2eRelevantPath admits all of apps/ and buildScripts/ on purpose. The comment on that predicate in classify-test-scope.yml reads: "this gate's failure mode is SILENCE. A false positive costs one ~3-minute job". Since #19705, a false positive costs 14 jobs.
  • The e2e launch chain draws buildScripts/ code from:
    • webpack.server.config.mjs (server-start);
    • the bundle-* and build-highlightjs builds;
    • e2eCiSelection.mjs;
    • the Playwright config, fixtures and specs. Their static imports stay inside build/, webpack/ and util/.
  • packageJsonRelevance() already reads package.json at both refs. package-lock.json admits components and e2e unconditionally.
    • A skills bump changes three entries in the lock: the root specifier, plus version, resolved and integrity on node_modules/neo-agent-skills.
  • test-e2e-brain.yml builds its matrix from e2e_brain_tier: one shard for smoke, ten for full. WhiteboxE2E.md:44-46 says the tier "never skips", and "Any other pull request runs the smoke subset".
  • The dev ruleset requires unit, components (1/3) to (3/3), e2e-engine, build-all and the baselines. It requires no e2e-brain context.

Design authority: the classifier comment quoted above prices a false positive at one ~3-minute job, which no longer holds. The Brain tier's "never skips" is my own AC on #19657, under #19655 (steward @neo-opus-ada). This change keeps that ticket's "always reports".

The Fix

In classify-test-scope.yml:

  1. isE2eRelevantPath drops apps/portal/resources/data/.
  2. It drops buildScripts/{create,docs,helpers,release}/. Because this is a deny-list, a new buildScripts/ directory still admits the tier.
  3. Packages that never reach a browser tier (neo-agent-skills) leave the dependency-kind comparison. package-lock.json gets the same both-refs read: once those packages' entries are removed, an unchanged lock admits nothing. An unreadable ref fails toward running.
  4. e2e_brain_tier gains none for a pull request outside the boundary. test-e2e-brain.yml then runs one shard whose only step reports the skip and its reason, so the context still reports. A push stays full.

Each change gets a red-first arm in WorkflowScopeClassifier.spec.mjs. WhiteboxE2E.md names the none tier.

Acceptance Criteria

  • #20321's file set (a learn/blog/*.md + blog.json) yields run_e2e=false, run_components=false, e2e_brain_tier=none and run_unit=true.
  • A file under buildScripts/create|docs|helpers|release/ alone runs no e2e. A file under buildScripts/build|webpack|util/, or under a new buildScripts/ directory, runs both tiers in full.
  • A bump of neo-agent-skills alone runs neither components nor e2e. The same bump plus any other lock change runs both, and so does an unreadable package-lock.json.
  • A docs-only PR's Brain job reports none with its reason and runs no Playwright. A push still selects full.
  • The 346-PR corpus, re-run against the changed script, admits the full tiers for fewer than 279 PRs, and every flip falls in one of the four rows above. The PR records this.
  • WhiteboxE2E.md states when the tier selects none.
  • The Brain job's skip step prints each changed file followed by in, or by the rule that left it out, passing the report through env. This was @neo-opus-ada's condition for none: a misclassification then shows on the PR that carries it.

Out of Scope

  • Spec-level selection inside the boundary (the breadth leaf of #20328).
  • Scoping unit, unit-tz, CodeQL or build-all.

Avoided Traps

  • Dropping all of apps/: the engine specs navigate more than 20 apps.
  • An allow-list of buildScripts/ files: such lists rot, as the classifier comment records.
  • Removing the Brain job on a skip: a reported skip costs one short job, while proving that no consumer reads the context costs more.

Related

#20328 (parent) · #20321 (witness) · #19655, #19657 (Brain tier) · #12705 (first docs-only scoping) · #18493 (shared classifier) · #18052 (classifier spec)

Sweeps: same as #20328, at 2026-11-05T14:21Z. They found no equivalent ticket and no prior exclusion. Origin Session ID: 8545cff9-f4d7-4731-b93e-7dfad1347c8a Retrieval Hint: "classifier misclassification portal data buildScripts create docs neo-agent-skills lock Brain tier none docs-only"