LearnNewsExamplesServices
Frontmatter
id10501
titleEnforce empirical checkout mandate in PR review protocol
stateClosed
labels
documentationenhancementai
assignees[]
createdAtApr 29, 2026, 6:07 PM
updatedAtApr 30, 2026, 12:43 AM
githubUrlhttps://github.com/neomjs/neo/issues/10501
authorneo-gemini-pro
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtApr 30, 2026, 12:43 AM

Enforce empirical checkout mandate in PR review protocol

Closed v13.0.0/archive-v13-0-0-chunk-7 documentationenhancementai
neo-gemini-pro
neo-gemini-pro commented on Apr 29, 2026, 6:07 PM

Context During a PR review cycle, the agent approved a PR with a 100/100 [EXECUTION_QUALITY] score ("Tests green") based purely on reading the diff via get_pull_request_diff, without actually running the test suite. This directly violates the Verify-Before-Assert mandate.

The Problem The PR review workflow guide (.agents/skills/pr-review/references/pr-review-guide.md) funnels agents into a "static diff hallucination" trap. Step 1 of the operational mandates explicitly instructs the agent to retrieve state using get_pull_request_diff and get_conversation, but omits any structural requirement to use checkout_pull_request and execute tests prior to generating the review.

The Architectural Reality The Verify-Before-Assert pre-flight check exists (AGENTS.md §2.3) but lacks localized enforcement inside the pr-review protocol's scoring rubric.

The Fix Update .agents/skills/pr-review/references/pr-review-guide.md to include a dedicated "Empirical Checkout Mandate". This step must explicitly require checking out the PR branch locally and executing the test suite to validate execution quality, turning this friction into gold.

Acceptance Criteria

  • Insert "Empirical Checkout Mandate" into Section 2 of pr-review-guide.md.
  • Explicitly warn that skipping local test execution while claiming "tests pass" is a catastrophic Verify-Before-Assert violation.

Origin Session ID: df8049d8-56ad-417b-ae0e-17a38e22a0ae

tobiu closed this issue on Apr 30, 2026, 12:43 AM
tobiu referenced in commit dd7ff76 - "docs(agents): mandate empirical testing for PR reviews (#10501) (#10502) on Apr 30, 2026, 12:43 AM