Context
#20328, cause 3: 20 job slots shared org-wide (the operator, 2026-11-05: "ORG wide PRs as the limitation"). In the replay corpus (346 PRs merged 10-13 to 11-05, classified by #20329's classifier), PRs start 903 jobs whose only work is to print that their tier is skipped. That averages 2.6 per PR, and a docs-only PR starts 11.
The Problem
When the classifier rules a tier out, its jobs still boot, check the gate, echo a reason and finish green:
- the three
components (n/3) shards;
- the four
e2e-engine (n/4) shards;
unit and the two unit-tz rows. These move out of scope: @neo-gpt-emmy's unit-family consolidation under #20328 folds all three into one job, and it lands first, so this leaf rebases onto it;
- since #20329, the one
e2e-brain job that reports none (mine).
Each of these holds a slot for its boot, though it does no testing.
The Architectural Reality
test.yml resolves the classifier's verdict into matrix.run and gates every step on it. test-e2e.yml gates every step on run_e2e and the live head. Both start the job either way.
- The
dev ruleset (19087298) requires unit, components (1/3), components (2/3), components (3/3) and the aggregate e2e-engine. unit-tz and e2e-brain are not required.
- GitHub reports a job skipped by
if: as Success, which satisfies a required context. A workflow-level path skip leaves the context pending ("Using conditions to control job execution"; "Troubleshooting required status checks").
Design authority: test-e2e.yml, comment above the skip step: "a job-level if yields a SKIPPED check … So the tier reports green with a stated reason on an irrelevant diff". That choice traded a runner for a readable reason. Under the slot cap the reason can move to the classifier job, which runs anyway, so the runner is no longer worth it.
The Fix
- The shard jobs take a job-level
if: from the classifier outputs and the head check, so no runner starts for a ruled-out tier.
- The reason moves into the classifier job's step summary, including #20329's per-file boundary report.
- A required context that names a shard becomes one aggregate job per tier, with
needs: plus if: always(), in the shape of e2e-engine. That is a ruleset edit, so it is the operator's and ships in the same change.
- The stale-head guard keeps protecting reruns.
- The Brain-tier workflow's
brain job, Resolve the Brain's dev tip, goes away: it boots a VM for a 3 s git ls-remote. The classifier job resolves the SHA and exposes it as an output, and skips the lookup when the tier is none. @neo-gemini-julia measured this and folded it in here (#20328 comment 6662846915), so the two leaves touching these files stay one owner's work.
Acceptance Criteria
Contract Ledger
| Target Surface |
Source of Authority |
Proposed Behavior |
Fallback |
Docs |
Evidence |
resolve_brain_tip (input of classify-test-scope.yml) |
this ticket |
Boolean, default false. Only test-e2e-brain.yml sets it to true. |
Unset: no tip step, and brain_sha is empty. test.yml and test-e2e.yml never set it. |
input description |
spec › "the classifier resolves the Brain tip only for a caller that asks and a tier that runs" |
brain_sha (output) |
this ticket |
The Brain's dev tip from git ls-remote. It is resolved only when the input is true and e2e_brain_tier != 'none'. |
Empty when not resolved. If git ls-remote returns nothing, test -n fails the classifier, so e2e-brain cannot run on an empty ref. That keeps actions/checkout from silently taking the default branch. |
output description + step comment |
consumers: the e2e-brain checkout ref and the receipt's BRAIN_SHA. Live on #20362: both read 5b042434f7. |
| Rerun boundary |
this ticket |
Re-running failed jobs reuses the attempt's classifier outputs, so it tests the same brain_sha. Re-running all jobs re-runs the classifier and resolves the newest tip. |
n/a |
tip step comment |
GitHub's rerun semantics; the step comment states it |
e2e-engine (the required context, e2e-engine-gate) |
dev ruleset 19087298 |
Passes for (classifier success, run_e2e=true, shards success) or (success, false, skipped). Everything else fails, including an empty or unknown run_e2e and a non-success classifier. |
n/a: fails closed |
gate comment |
spec › "the required e2e-engine context passes a skipped tier only when the classifier ruled it out" runs the script under bash -e across nine cases |
| Classifier step summary |
this ticket |
Written in every caller whenever run_e2e is false or the Brain tier is none: the reason plus each changed file's boundary rule. |
Not written when every tier runs |
WhiteboxE2E.md "selects none" |
spec › "a ruled-out tier writes its reason and each file's boundary rule to the classifier summary" |
e2e_boundary_report (output) |
#20329 |
Unchanged. Its one workflow consumer, the Brain skip step, is retired. The summary now prints the same string. |
n/a |
#20329's ledger |
— |
Out of Scope
- The three
components shards (narrowed out 2026-11-05):
- The ruleset requires them by shard name, and a matrix skipped by a job-level
if does not report those names.
- They need an aggregate
components gate plus the operator's ruleset swap. That is the same aggregate-context decision as neomjs/neo-agent-skills#281, so it ships as its own leaf once the operator decides.
- What runs when a tier is in scope (#20329, #20330).
- The shared PR baseline (neomjs/neo-agent-skills#281).
- Merging the three classifier calls into one workflow, which would undo the separate e2e pipeline the operator set in #18408.
Avoided Traps
- Workflow-level
paths: leaves required contexts pending.
- Dropping the reason: a skipped tier would then look like an untested one, so the reason moves to the classifier's summary rather than going away.
Related
#20328 (parent) · #20329 (classifier, boundary report) · neomjs/neo-agent-skills#281 (baseline) · #18408 (separate e2e pipeline) · #17171 (lint contexts)
Sweeps at 2026-11-05T15:19Z found no equivalent:
- latest open;
- an all-state search on job-level / skip job / runner slot / aggregate context;
- A2A claims;
- Memory Core.
unowned-rationale: it depends on #20332 merging (it rewrites that PR's none job) and on the operator's ruleset edit. I'll take it once #20332 merges, unless a peer claims it first.
Origin Session ID: 8545cff9-f4d7-4731-b93e-7dfad1347c8a
Retrieval Hint: "skip-only jobs job-level if required context aggregate runner slots components shards e2e-engine"
Context
#20328, cause 3: 20 job slots shared org-wide (the operator, 2026-11-05: "ORG wide PRs as the limitation"). In the replay corpus (346 PRs merged 10-13 to 11-05, classified by #20329's classifier), PRs start 903 jobs whose only work is to print that their tier is skipped. That averages 2.6 per PR, and a docs-only PR starts 11.
The Problem
When the classifier rules a tier out, its jobs still boot, check the gate, echo a reason and finish green:
components (n/3)shards;e2e-engine (n/4)shards;unitand the twounit-tzrows. These move out of scope: @neo-gpt-emmy's unit-family consolidation under #20328 folds all three into one job, and it lands first, so this leaf rebases onto it;e2e-brainjob that reportsnone(mine).Each of these holds a slot for its boot, though it does no testing.
The Architectural Reality
test.ymlresolves the classifier's verdict intomatrix.runand gates every step on it.test-e2e.ymlgates every step onrun_e2eand the live head. Both start the job either way.devruleset (19087298) requiresunit,components (1/3),components (2/3),components (3/3)and the aggregatee2e-engine.unit-tzande2e-brainare not required.if:as Success, which satisfies a required context. A workflow-level path skip leaves the context pending ("Using conditions to control job execution"; "Troubleshooting required status checks").Design authority:
test-e2e.yml, comment above the skip step: "a job-levelifyields a SKIPPED check … So the tier reports green with a stated reason on an irrelevant diff". That choice traded a runner for a readable reason. Under the slot cap the reason can move to the classifier job, which runs anyway, so the runner is no longer worth it.The Fix
if:from the classifier outputs and the head check, so no runner starts for a ruled-out tier.needs:plusif: always(), in the shape ofe2e-engine. That is a ruleset edit, so it is the operator's and ships in the same change.brainjob,Resolve the Brain's dev tip, goes away: it boots a VM for a 3 sgit ls-remote. The classifier job resolves the SHA and exposes it as an output, and skips the lookup when the tier isnone. @neo-gemini-julia measured this and folded it in here (#20328 comment 6662846915), so the two leaves touching these files stay one owner's work.Acceptance Criteria
e2e-engineshard and noe2e-brainjob starts a runner. The classifier summary carries the skip reason and the boundary report.e2e-enginecontext still reports on such a PR. It passes a skip only when the classifier itself succeeded with an explicitrun_e2e=false.brainjob starts. The Braindevtip arrives as a classifier output. Every shard of a run attempt, and every rerun of its failed jobs, tests that same SHA, and the receipt records it.Contract Ledger
resolve_brain_tip(input ofclassify-test-scope.yml)false. Onlytest-e2e-brain.ymlsets it totrue.brain_shais empty.test.ymlandtest-e2e.ymlnever set it.descriptionbrain_sha(output)devtip fromgit ls-remote. It is resolved only when the input istrueande2e_brain_tier != 'none'.git ls-remotereturns nothing,test -nfails the classifier, soe2e-braincannot run on an empty ref. That keepsactions/checkoutfrom silently taking the default branch.description+ step commente2e-braincheckoutrefand the receipt'sBRAIN_SHA. Live on #20362: both read5b042434f7.brain_sha. Re-running all jobs re-runs the classifier and resolves the newest tip.e2e-engine(the required context,e2e-engine-gate)devruleset19087298success,run_e2e=true, shardssuccess) or (success,false,skipped). Everything else fails, including an empty or unknownrun_e2eand a non-success classifier.bash -eacross nine casesrun_e2eis false or the Brain tier isnone: the reason plus each changed file's boundary rule.e2e_boundary_report(output)Out of Scope
componentsshards (narrowed out 2026-11-05):ifdoes not report those names.componentsgate plus the operator's ruleset swap. That is the same aggregate-context decision as neomjs/neo-agent-skills#281, so it ships as its own leaf once the operator decides.Avoided Traps
paths: leaves required contexts pending.Related
#20328 (parent) · #20329 (classifier, boundary report) · neomjs/neo-agent-skills#281 (baseline) · #18408 (separate e2e pipeline) · #17171 (lint contexts)
Sweeps at 2026-11-05T15:19Z found no equivalent:
unowned-rationale: it depends on #20332 merging (it rewrites that PR's
nonejob) and on the operator's ruleset edit. I'll take it once #20332 merges, unless a peer claims it first. Origin Session ID: 8545cff9-f4d7-4731-b93e-7dfad1347c8a Retrieval Hint: "skip-only jobs job-level if required context aggregate runner slots components shards e2e-engine"