LearnNewsExamplesServices
Frontmatter
id11465
titleAdd CI fail-fast gate before formal PR review
stateClosed
labels
documentationaiagent-task:in-progressarchitecturemodel-experience
assigneesneo-gpt
createdAtMay 16, 2026, 4:33 PM
updatedAtMay 16, 2026, 5:00 PM
githubUrlhttps://github.com/neomjs/neo/issues/11465
authorneo-gpt
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtMay 16, 2026, 5:00 PM

Add CI fail-fast gate before formal PR review

Closeddocumentationaiagent-task:in-progressarchitecturemodel-experience
neo-gpt
neo-gpt commented on May 16, 2026, 4:33 PM

Context

This ticket comes from live PR-review friction on 2026-05-16:

  • PR #11462 initially had a failing lint check. Opus posted a full CHANGES_REQUESTED review that included the CI failure plus other findings.
  • Gemini then requested GPT review while a fresh check run was still pending.
  • @tobiu sharpened the rule: a CI failure should enforce no additional review needed until the author fixes it, and we can go stronger — no formal review at all, even Cycle 1, while required CI is red.

This is a cost-saving MX rule, not a new quality criterion. CI already tells the author the PR is not mergeable. Spending a full human/agent review cycle while required checks are failing creates review churn, stale findings, and duplicate Required Actions.

The Problem

Current pr-review substrate forbids approval while CI is failing or pending, but it still encourages the reviewer to produce a formal review and list failing checks as Required Actions:

  • .agents/skills/pr-review/audits/ci-security-audit.md says pending checks mean the reviewer must hold and cannot approve.
  • The same audit says failing checks should be flagged as Required Actions.
  • The templates include a CI audit section, but they do not define a pre-review fail-fast exit.

That means agents can still spend a full Cycle 1 or Cycle N review on a PR whose required CI status already blocks merge. This was visible in #11462: once lint was red, the cheapest correct action was to return the PR to the author immediately, not stack additional formal review work.

The Architectural Reality

The correct substrate is the existing CI audit payload:

  • pr-review-guide.md §7.6 already maps CI/security checking to .agents/skills/pr-review/audits/ci-security-audit.md.
  • ci-security-audit.md is the granular World Atlas payload for this rule family.
  • Adding this to the main pr-review-guide.md would bloat the map; the guide already delegates CI detail to the audit file.

Related but not duplicate:

  • Discussion #11237 covers mechanical enforcement against invalid approvals while CI is failing.
  • This ticket covers reviewer behavior before formal review is spent: fail fast and defer until the author produces a fresh green/passing check state, unless the author explicitly asks for CI triage help.

The Fix

Update .agents/skills/pr-review/audits/ci-security-audit.md with a small pre-review gate:

  1. Run gh pr checks <N> before reading/drafting a substantive review.
  2. If any required check is failing, do not post a formal PR review.
  3. Send a lightweight A2A / PR comment deferral naming the failing check(s) and ask the author to fix CI first.
  4. If checks are pending/in-progress, hold the review until they finish.
  5. Exception: if the author explicitly asks for CI triage or the failure is plausibly infrastructure/flaky and reviewer help is the task, the reviewer may perform a CI-triage comment, but should not score the full diff or issue a formal approval/request-changes review until checks are green or the failure is explicitly classified.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
pr-review CI audit payload @tobiu 2026-05-16 review-friction directive + #11462 live friction Required CI red => no formal review cycle; author fixes first CI-triage-only comment when explicitly requested or infrastructure failure suspected .agents/skills/pr-review/audits/ci-security-audit.md Local grep verifies guide still points to audit; focused lint/grep verifies wording
Reviewer A2A behavior AGENTS.md §0 mandatory A2A + pr-review §10 handoff discipline Send lightweight deferral instead of full formal review If no A2A target exists, comment on PR with CI-deferral note Audit payload text Manual review of inserted rule

Acceptance Criteria

  • ci-security-audit.md contains an explicit pre-review fail-fast gate for failing required CI.
  • The rule says formal review should not be spent while CI is red, including Cycle 1.
  • Pending/in-progress checks are handled as a hold, not as a formal review trigger.
  • A narrow exception exists for explicit CI-triage requests / plausible infrastructure failures.
  • No main pr-review-guide.md body bloat beyond existing pointer structure.
  • Verification includes node ai/scripts/lint-skill-manifest.mjs --base origin/dev or a documented reason if current unmerged skill-lint work makes that unavailable.

Out of Scope

  • Mechanical GitHub Actions enforcement from Discussion #11237.
  • Changing PR branch protection.
  • Dismissing existing reviews.
  • Rewriting review templates beyond what is strictly needed for this fail-fast gate.
  • Deciding which checks are required vs optional; the gate consumes current gh pr checks / branch-protection reality.

Avoided Traps

  • Trap: formal Request Changes for every red CI run. Rejected because CI itself is already the blocking signal; duplicate formal reviews create re-review churn.
  • Trap: bury the rule in pr-review-guide.md. Rejected because CI detail already lives in the audit payload.
  • Trap: forbid all CI investigation. Rejected; explicit CI-triage requests and suspected infrastructure failures still need reviewer help.

Related

  • PR #11462 — live friction anchor
  • Discussion #11237 — adjacent approval-enforcement discussion
  • .agents/skills/pr-review/audits/ci-security-audit.md
  • .agents/skills/pr-review/references/pr-review-guide.md §7.6

Origin Session ID: 6ec143cb-2e5b-4964-94d6-eb28cb25bde2 Retrieval Hint: "CI fail-fast gate before formal PR review #11462 lint failure"

tobiu referenced in commit 8d0f677 - "docs(pr-review): add CI fail-fast review gate (#11465) (#11466) on May 16, 2026, 5:00 PM
tobiu closed this issue on May 16, 2026, 5:00 PM