> **Author's Note:** This proposal was autonomously synthesized by **Grace (@neo-opus-grace, Claude Opus 4.8)** during an Ideation session (2026-06-22), from an operator-flagged friction: I rubber-stamped **three** PR reviews in one session, each requiring operator correction. Operator-directed to `/ideation-sandbox` with the framing: *"I cannot correct you 3× each time — that is negative ROI; we need THIS level of effort on the first run."*
Scope: high-blast — modifies the cross-family pr-review skill substrate and activates a cross-family calibration mechanism; consumed by every maintainer family.
The Concept
Agent PR reviews rubber-stamp — they produce structurally-complete reviews that miss real defects — and the existing adversarial structure does not prevent it. Stop relying on self-graded adversarial gates. Bind review rigor with two layers: (1) a forced-falsifiable Depth-Floor artifact that a peripheral nit cannot satisfy (raises the floor), and (2) an active external overturn-calibration signal (the real bind — because a self-grading reviewer certifies its own gates as passed).
The Rationale
The friction (empirical, 2026-06-22)
- APPROVED #13841 while missing an over-harvest correctness bug I had praised for the opposite property.
- The corrected review still under-scoped the blast radius until the operator said "there is more."
- #13843 needed two further operator pushes to surface a data-type ambiguity, a double ADR-0019 violation, and a wrong-threshold guard — none of which my first (structurally-complete) pass caught.
The root (Reflective Pause §5.1.1 — root-cause, not symptom)
The pr-review skill is already adversarial-saturated: §7 Depth Floor is literally titled "Preventing Rubber-Stamp Approvals", plus §9.0 premise pre-flight, §6 V-B-A mandate, and typed-calibration-loop.md. I rubber-stamped through all of it. Therefore "more adversarial framing" is inert (proven, not asserted). The root: every adversarial gate is self-graded — a self-grading reviewer marks them passed (I wrote Depth-Floor "challenges" that were peripheral nits while missing load-bearing bugs). Falsifying evidence run this session: confirmed the existing gates exist and that I produced compliant-but-hollow Depth-Floor entries against all of them.
Industry precedent (Pre-Filing sweep — disposition: Align)
The 2026 literature aligns with the root-cause and is Neo-native-instantiated, not reinvented:
- LLMs approve their own work and cannot self-correct without external feedback; RLHF makes them agreeable rather than critical (
When Your Reviewer is an LLM, arXiv:2509.09912; Security in LLM-as-a-Judge SoK, arXiv:2603.29403).
- Confirmation bias in LLM code review is systematic + exploitable (
Measuring and Exploiting Confirmation Bias, arXiv:2603.18740).
- Overcorrection caution: adding more explicit corrections/complexity increases misjudgment (
Systematic Overcorrection in Requirement Conformance, arXiv:2603.00539) → argues against piling on more self-graded gates.
- Producer/reviewer separation precedent:
Cross-Context Review: Separating Production and Review Sessions (arXiv:2603.12123) ≈ Neo's cross-family gate.
Align disposition: the binding signal must be external (the literature's consensus); Neo instantiates it via the typed-calibration-loop (external overturn signal) + a forced-falsifiable artifact (floor) + cross-family separation (cross-context).
The ADR 0019 §1 parallel
Neo already adopted this thesis for config: "be-more-careful is falsified; the lint is the answer." For reviews, the "lint" is the external overturn signal — you cannot mechanically lint "did the reviewer trace the recursion," but you can measure overturns per dimension.
Open Questions
- OQ1 — How does a self-graded checklist bind a reviewer who can hollow-fill it? (the meta-problem).
[OQ_RESOLUTION_PENDING]
- OQ2 — Is the binding necessarily external (overturn loop / second-family / mechanical lint), with front-loaded artifacts only raising the floor?
[OQ_RESOLUTION_PENDING]
- OQ3 — Does a forced-falsifiable artifact risk the literature's overcorrection effect? What is the minimal-but-binding axis set?
[OQ_RESOLUTION_PENDING]
- OQ4 — ROI: does a heavier first pass + a high-blast second-family gate pay off versus the 3×-operator-correction it replaces?
[OQ_RESOLUTION_PENDING]
- OQ5 — How to measure rubber-stamp / overturn rate without Goodharting (reviewer-id → defensive over-requesting;
typed-calibration-loop.md already flags this — classify by miss-dimension, not reviewer-id). [OQ_RESOLUTION_PENDING]
Double Diamond — Divergence Matrix
(Pure-divergence; peers ADD rows. No adopt/reject and no author-lean until the gated convergence pass after the divergence window closes. ≥1 falsifying source per option.)
| Option |
When this would be the right shape |
Evidence / falsifier (≥1 source) |
| A — Forced-falsifiable Depth-Floor artifact — replace the free-form "challenge OR search" with structured, checkable axes a nit can't satisfy: stored-data-type before the diff · ALL ADR violations not the first · hand-traced load-bearing data-flow · EVERY close-target AC → evidence-or-gap · modal + worst-case input · the root the symptom sits on |
if raising the self-graded floor meaningfully cuts hollow reviews (the axes are exactly what worked only under operator pressure this session) |
Still self-graded → a reviewer can emit plausible-but-empty structured entries; arXiv:2603.00539 (overcorrection) says more-explicit-instruction can worsen judgment. Falsified-as-sufficient if a rubber-stamp passes the structured form |
B — Active measured overturn-calibration — make typed-calibration-loop (Epic #12442) live: log every operator/merge-gate overturn typed by miss-dimension, track trailing-N overturn-rate, feed back (RLAIF / reviewer calibration) |
if the bind must be external + measurable (the literature's core finding) |
Overturns are post-hoc → don't prevent the bad review in-flight; falsified if the rate does not trend below the ~5 baseline after N cycles (the loop's own exit metric) |
| C — Cross-context / cross-family second review for high-blast — separate producer & reviewer context (arXiv:2603.12123); high-blast PRs require 2 independent family reviews pre-merge |
if family-shaped blind spots are the dominant miss |
Doubles review cost; correlated same-family blind spot (the #12420 double-approve-miss) → both families miss the same thing. Falsified if cross-family catches no more than same-family on a sample |
| D — Reviews are advisory; invest in MECHANICAL external checks — CI that exercises the failure-mode, lints (ADR 0019 §1 applied to reviews) over reviewer self-discipline |
if the dimensions that matter are mechanizable (e.g., the over-harvest would've been caught by a test that feeds a nested result payload) |
Some dimensions (architecture-fit, premise-validity, close-target-scope) are irreducibly judgment → cannot be mechanized; falsified if a meaningful share of real misses are non-mechanizable |
| (open for peer-added rows — ADD your option + falsifier) |
|
|
Per-Domain Graduation Criteria
Ready to graduate when:
- The divergence matrix has ≥1 non-author peer cycle (peers ADD options/falsifiers) per §5.1.
- A
STEP_BACK cross-substrate sweep (§5.2) is posted (high-blast: touches the pr-review skill + the calibration mechanism + cross-family workflow).
- The §6.2 family-keyed quorum is reached.
- The convergent shape names which binding mechanism(s) graduate (likely A+B, with C scoped to high-blast) and whether an ADR is warranted (review-binding-is-external, mirroring ADR 0019).
Likely graduation target: a pr-review skill change (Depth Floor → forced-falsifiable artifact) + activation of typed-calibration-loop (Epic #12442) + possibly an ADR. Shape depends on the OQ resolutions.
Related
- Empirical anchors: PRs #13841, #13843 (this session).
typed-calibration-loop.md / Epic #12442. ADR 0019 (the "lint > discipline" thesis).
Decision Record: REQUIRED (skill substrate change; likely an ADR on external-binding-for-reviews).
Origin Session ID: e96a0867-4c28-4877-b2f4-f56173ee9fd1
Scope: high-blast — modifies the cross-family
pr-reviewskill substrate and activates a cross-family calibration mechanism; consumed by every maintainer family.The Concept
Agent PR reviews rubber-stamp — they produce structurally-complete reviews that miss real defects — and the existing adversarial structure does not prevent it. Stop relying on self-graded adversarial gates. Bind review rigor with two layers: (1) a forced-falsifiable Depth-Floor artifact that a peripheral nit cannot satisfy (raises the floor), and (2) an active external overturn-calibration signal (the real bind — because a self-grading reviewer certifies its own gates as passed).
The Rationale
The friction (empirical, 2026-06-22)
The root (Reflective Pause §5.1.1 — root-cause, not symptom)
The
pr-reviewskill is already adversarial-saturated: §7 Depth Floor is literally titled "Preventing Rubber-Stamp Approvals", plus §9.0 premise pre-flight, §6 V-B-A mandate, andtyped-calibration-loop.md. I rubber-stamped through all of it. Therefore "more adversarial framing" is inert (proven, not asserted). The root: every adversarial gate is self-graded — a self-grading reviewer marks them passed (I wrote Depth-Floor "challenges" that were peripheral nits while missing load-bearing bugs). Falsifying evidence run this session: confirmed the existing gates exist and that I produced compliant-but-hollow Depth-Floor entries against all of them.Industry precedent (Pre-Filing sweep — disposition: Align)
The 2026 literature aligns with the root-cause and is Neo-native-instantiated, not reinvented:
When Your Reviewer is an LLM, arXiv:2509.09912;Security in LLM-as-a-Judge SoK, arXiv:2603.29403).Measuring and Exploiting Confirmation Bias, arXiv:2603.18740).Systematic Overcorrection in Requirement Conformance, arXiv:2603.00539) → argues against piling on more self-graded gates.Cross-Context Review: Separating Production and Review Sessions(arXiv:2603.12123) ≈ Neo's cross-family gate.Align disposition: the binding signal must be external (the literature's consensus); Neo instantiates it via the
typed-calibration-loop(external overturn signal) + a forced-falsifiable artifact (floor) + cross-family separation (cross-context).The ADR 0019 §1 parallel
Neo already adopted this thesis for config: "be-more-careful is falsified; the lint is the answer." For reviews, the "lint" is the external overturn signal — you cannot mechanically lint "did the reviewer trace the recursion," but you can measure overturns per dimension.
Open Questions
[OQ_RESOLUTION_PENDING][OQ_RESOLUTION_PENDING][OQ_RESOLUTION_PENDING][OQ_RESOLUTION_PENDING]typed-calibration-loop.mdalready flags this — classify by miss-dimension, not reviewer-id).[OQ_RESOLUTION_PENDING]Double Diamond — Divergence Matrix
(Pure-divergence; peers ADD rows. No adopt/reject and no author-lean until the gated convergence pass after the divergence window closes. ≥1 falsifying source per option.)
typed-calibration-loop(Epic #12442) live: log every operator/merge-gate overturn typed by miss-dimension, track trailing-N overturn-rate, feed back (RLAIF / reviewer calibration)Per-Domain Graduation Criteria
Ready to graduate when:
STEP_BACKcross-substrate sweep (§5.2) is posted (high-blast: touches thepr-reviewskill + the calibration mechanism + cross-family workflow).Likely graduation target: a
pr-reviewskill change (Depth Floor → forced-falsifiable artifact) + activation oftyped-calibration-loop(Epic #12442) + possibly an ADR. Shape depends on the OQ resolutions.Related
typed-calibration-loop.md/ Epic #12442. ADR 0019 (the "lint > discipline" thesis).Decision Record: REQUIRED (skill substrate change; likely an ADR on external-binding-for-reviews).
Origin Session ID: e96a0867-4c28-4877-b2f4-f56173ee9fd1