Context
Neo's PR-review execution rule was created after the static-diff hallucination in #10501: a reviewer asserted that tests were green without executing them. That correction was right for its time, but it predates the current CI matrix and later CI-first review controls.
The substrate has since changed materially:
#10897 added the unit + integration CI matrix.#11465 made green CI a precondition for spending a formal review.#11467 delayed actionable author-side review requests until current-head CI is green.#11441 and #11885 established review-cost reduction as an explicit MX concern.
The remaining rule still requires every code reviewer to check out the branch and discover/run related tests locally. On 2026-07-13, operator review-pressure analysis challenged that rule: reviewers generally cannot know which tests a change affects without scanning the test tree and reconstructing dependency reach. That discovery can cost more than the duplicate execution it is meant to justify.
Live latest-open sweep: checked the latest 20 open issues plus scoped live search and recent A2A claims immediately before creation; no equivalent ticket or in-flight claim exists.
The Problem
The current pr-review contract conflates three different responsibilities:
- CI executing routine unit/integration coverage.
- Authors proving behavior that hosted CI does not execute.
- Reviewers falsifying concrete behavioral concerns.
As a result, reviewers are told to rediscover related tests and rerun them even when exact-head CI already executed the canonical suite. The inverse shortcut is also unsafe: author evidence cannot trigger only when a PR claims whitebox E2E or another non-CI evidence class, because an author could simply omit the claim. A change to an application or feature with existing non-CI coverage can affect that coverage even when the test file is untouched.
The current cold-review guide + Cycle-1 template measure 50,160 bytes before PR-specific context. Adding a new decision table, static NEO_TEST_SKIP_CI file list, classifier ritual, or audit payload would increase the very review pressure this ticket must reduce.
The Architectural Reality
This is one test-evidence field-class correction across two existing skills:
- .agents/skills/pr-review/references/pr-review-guide.md owns reviewer execution behavior.
- .agents/skills/pr-review/assets/pr-review-template.md and pr-review-followup-template.md own review evidence shape.
- .agents/skills/pull-request/references/pull-request-workflow.md already requires every agent PR body to contain ## Test Evidence; it owns author evidence.
- Required exact-head CI owns routine unit/integration execution.
- Existing non-CI coverage of a touched surface—not author prose—triggers author-side evidence.
- Reviewers own targeted falsifiers when the review surfaces a named behavioral concern.
- Test-location review remains relevant only when a test is added or moved.
Progressive Disclosure outcome: modify and compress existing payloads. Do not add a skill, audit payload, static skip registry, or always-loaded rule.
Structure-map gate: no new .mjs file or novel code placement is involved; the owning substrate is the existing pr-review and pull-request skill payloads.
The Fix
Replace the unconditional reviewer-local-rerun rule with this boundary:
Required exact-head CI is the default execution evidence for routine unit/integration behavior. Reviewers do not search for or rerun related unit/integration tests solely to duplicate green CI. A reviewer runs a targeted test only when the review identifies a named behavior not established by CI, and records that falsifier. Authors execute existing non-CI coverage for every touched surface and record the receipt in ## Test Evidence.
Apply it coherently:
- Rewrite pr-review-guide.md §2.2 and §7.5 to preserve exact-head source inspection while removing mandatory reviewer discovery/reruns.
- Replace the Cycle-1 template's local-run checklist with terse exact-head CI + optional named-falsifier fields.
- Align the follow-up template with the same boundary.
- Extend author-side ## Test Evidence guidance: if a touched application/feature has existing whitebox E2E, component, visual, local-model, embedding, host-native, or other non-CI coverage, the author runs the relevant existing coverage even when no test file changed and no evidence class is claimed.
- Keep test-placement validation only for added/moved tests.
- Rewrite the §7.7 anti-pattern so the violation is claiming execution without exact-head CI/author evidence/falsifier—not failing to duplicate the author's command.
- Net-reduce the loaded review surface; do not create another payload.
Contract Ledger Matrix
| Target Surface |
Source of Authority |
Proposed Behavior |
Fallback |
Docs |
Evidence |
| Reviewer routine execution evidence |
Current required CI + #10501 anti-hallucination lineage + #11465 CI-first review gate |
Exact-head green required CI is sufficient; no default local rerun or test-tree discovery |
Named reviewer falsifier when a concrete behavior remains uncertain |
pr-review-guide.md §2.2/§7.5 |
Exact wording audit + byte measurement + skill lint |
| Reviewer formal evidence shape |
pr-review-template.md / follow-up template |
Record exact-head CI and optional targeted falsifier; test location only for added/moved tests |
Full evidence note when CI does not establish the challenged behavior |
Existing templates |
Template validation / representative dry run |
| Author non-CI evidence |
pull-request-workflow.md Test Evidence contract |
Existing non-CI coverage for every touched surface is executed and named, independent of PR claims or test-file changes |
Explicit environment-bound residual with post-merge/live validation; never silent omission |
pull-request-workflow.md |
PR-body example + source audit |
| Skill loaded surface |
ADR 0008 Map/Atlas + substrate-accretion defense |
Compress existing payloads; no new audit/list/classifier prose |
None |
Existing payloads only |
wc -c before/after + lint-skill-manifest.mjs |
Acceptance Criteria
Decision Record impact
Aligned with ADR 0008 (existing payload compression and decision-atom refinement; no new primitive or ADR amendment).
Out of Scope
- Adding whitebox E2E, component, or visual suites to hosted CI;
#15110 is the adjacent CI-coverage problem.
- Resolving or reclassifying current NEO_TEST_SKIP_CI guards.
- Building a test dependency graph or reviewer-side coverage classifier.
- Weakening patch-blind premise review, architecture review, security review, or targeted behavioral falsification.
- Treating green CI as approval.
- Changing cross-family review or human-only merge authority.
Avoided Traps
- Reviewer test-tree archaeology: rejected because the reviewer lacks implementation-local knowledge and can spend more context discovering tests than the rerun adds.
- Evidence-by-author-claim: rejected because omitting the claim would bypass existing non-CI coverage.
- Static skip-file list: rejected because NEO_TEST_SKIP_CI mixes environment requirements, state pollution, races, and missing fixtures; it is neither a safe-run registry nor dependency map.
- New audit payload: rejected because the rule is a compact responsibility boundary and must net-reduce review substrate.
- Green CI equals correctness: rejected; CI is execution evidence, while reviewers still own premise, architecture, diff correctness, and named falsifiers.
Related
Origin Session ID: 837ad74b-c2d2-413d-9aab-b7165a93a82a
Retrieval Hint: "PR review pressure CI default execution evidence author existing non-CI coverage touched surface targeted reviewer falsifier"
Context
Neo's PR-review execution rule was created after the static-diff hallucination in
#10501: a reviewer asserted that tests were green without executing them. That correction was right for its time, but it predates the current CI matrix and later CI-first review controls.The substrate has since changed materially:
#10897added the unit + integration CI matrix.#11465made green CI a precondition for spending a formal review.#11467delayed actionable author-side review requests until current-head CI is green.#11441and#11885established review-cost reduction as an explicit MX concern.The remaining rule still requires every code reviewer to check out the branch and discover/run related tests locally. On 2026-07-13, operator review-pressure analysis challenged that rule: reviewers generally cannot know which tests a change affects without scanning the test tree and reconstructing dependency reach. That discovery can cost more than the duplicate execution it is meant to justify.
Live latest-open sweep: checked the latest 20 open issues plus scoped live search and recent A2A claims immediately before creation; no equivalent ticket or in-flight claim exists.
The Problem
The current pr-review contract conflates three different responsibilities:
As a result, reviewers are told to rediscover related tests and rerun them even when exact-head CI already executed the canonical suite. The inverse shortcut is also unsafe: author evidence cannot trigger only when a PR claims whitebox E2E or another non-CI evidence class, because an author could simply omit the claim. A change to an application or feature with existing non-CI coverage can affect that coverage even when the test file is untouched.
The current cold-review guide + Cycle-1 template measure 50,160 bytes before PR-specific context. Adding a new decision table, static NEO_TEST_SKIP_CI file list, classifier ritual, or audit payload would increase the very review pressure this ticket must reduce.
The Architectural Reality
This is one test-evidence field-class correction across two existing skills:
Progressive Disclosure outcome: modify and compress existing payloads. Do not add a skill, audit payload, static skip registry, or always-loaded rule.
Structure-map gate: no new .mjs file or novel code placement is involved; the owning substrate is the existing pr-review and pull-request skill payloads.
The Fix
Replace the unconditional reviewer-local-rerun rule with this boundary:
Apply it coherently:
Contract Ledger Matrix
#10501anti-hallucination lineage +#11465CI-first review gateAcceptance Criteria
Decision Record impact
Aligned with ADR 0008 (existing payload compression and decision-atom refinement; no new primitive or ADR amendment).
Out of Scope
#15110is the adjacent CI-coverage problem.Avoided Traps
Related
Origin Session ID: 837ad74b-c2d2-413d-9aab-b7165a93a82a
Retrieval Hint: "PR review pressure CI default execution evidence author existing non-CI coverage touched surface targeted reviewer falsifier"