LearnNewsExamplesServices
Frontmatter
id17527
titleThe PR body certifies every close-target AC with evidence, and the body lint enforces it — paid for in payload bytes
stateClosed
labels
enhancementaitestingmodel-experience
assigneesneo-fable-clio
createdAtAug 22, 2026, 1:51 AM
updatedAtAug 22, 2026, 3:13 AM
githubUrlhttps://github.com/neomjs/neo/issues/17527
authorneo-fable-clio
commentsCount3
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 22, 2026, 3:13 AM

The PR body certifies every close-target AC with evidence, and the body lint enforces it — paid for in payload bytes

Closed Backlog/active-chunk-18 enhancementaitestingmodel-experience
neo-fable-clio
neo-fable-clio commented on Aug 22, 2026, 1:51 AM

Problem

AC verification is structurally reviewer-only today. pull-request-workflow.md:337 assigns it outright ("pr-review checks close-target ACs"), the author-side AC re-anchor (line 123) is prose discipline, and agent-pr-body-lint.yml enforces section presence (Evidence:, ## Test Evidence, ## Post-Merge Validation, ## Deltas) — never AC coverage; the lint cannot know how many ACs the close target carries. Consequence, operator-observed: a peer can knowingly skip ticket ACs and nothing before human/reviewer eyes can tell whether an implementation even roughly followed plan.

A second, adjacent debt (operator-named): the body's test-evidence ceremony is pre-CI-era thinking. What CI executes itself needs no author testimony — red CI means no review, so a body listing CI-covered runs is redundant bytes. Author testimony is owed ONLY for evidence that lives outside CI (e2e journeys, visual pins, live verification) — exactly the class where "the AC-carrying spec was never executed" (PR #17510's RA-1) can hide today.

A third debt, same root (the process form does not scale with the work unit): the review side has exactly ONE round-1 form — the full template. Measured on PR #17528 (+11/−5, a bounded repair whose prescription pre-existed in the reopen): the full form cost the reviewer 8m18s wall plus a delegated independent-audit subagent, to re-validate architecture nobody changed. At ~700 PRs/month this fixed cost IS the PR-creation→approval bottleneck; per-reviewer subagent delegation scales the cost into tokens instead of removing it. The v13.0 A+FU experiment already proved the wrong answer (defer findings into new full-cost units → micro-ticket flood); the right answer is a sanctioned SMALLER REVIEW FORM with mechanical eligibility — authors and reviewers must both know what a micro review is and when to use it.

Fresh data (RA-class sweep, 55 required actions across 22 of the last 30 merged dev PRs, 2026-08-21): ~28% of RAs were genuine behavior breaks concentrated in fleet-critical zones, ~18% test-integrity catches CI is structurally blind to, ~33% ledger/prose — the classes inform both the AC gate and the micro-review eligibility bounds below.

Fix

A — Author side: the AC self-certificate

  1. Template — AC Evidence table: the PR-body template gains ## AC Evidence — one row per close-target AC: AC → proof. Proof forms differ by coverage class: for a CI-covered AC the proof is the owning spec reference (CI supplies the green; no command, no run log). For an outside-CI AC (e2e journey, visual pin, live verification) the proof carries command + receipt, because no machine enforces it. Close targets without a structured AC list declare No structured ACs on #N instead. Ticket ACs are matched by count in order (the dominant ticket format is anonymous - [ ] checkbox lines; numbered AC-N: lines also count) — no ticket migration.
  2. Lint: agent-preflight.mjs's validatePrBody (the one owning implementation CI already runs) gains the AC-coverage gate via an injectable close-target-AC resolver (same pattern as resolveOwnerState): it resolves the close target from Resolves #N, reads the ticket body, extracts its structured AC lines, and fails when the table's row count misses the ticket's AC count or a row's proof slot is empty. The No structured ACs declaration passes only when the ticket really has no structured AC list. Unreadable ticket → warning, never a verdict.
  3. Test-evidence diet (operator rule: "don't mention tests at all, unless outside of CI"): the ## Test Evidence section's contract shrinks to outside-CI evidence only; when everything runs in CI, the section is one line ("All coverage runs in CI."). The section anchor stays; the ceremony goes.

B — Reviewer side: the micro review (operator-directed fold-in)

Discovery (V-B-A during implementation): the micro review already SHIPS — PullRequestService.mjs validates # PR Micro-Review (anchors **Class:** micro|contained|mechanical, **Verdict:**, **Glance:**; wired at the validation entry) and agent-pr-review-body-lint.yml mirrors it with a sync comment. What was missing is exactly the operator's sentence: NOBODY COULD KNOW — the service cites "pr-review-guide §7 blast-scaling", the guide contained the word "micro" zero times, no template asset existed, and the author side had no signal. So part B builds NO server code:

  1. Documentation of the shipped form: pr-review-guide.md gains §6.4 — the form, the shipped eligibility semantics (mechanical = no architectural concept, ANY size; micro/contained diffs; NEVER for ADR / new abstraction / consumed contract / security / migration / fleet-critical zones), and the three-axes disambiguation against §6.1's micro-change MERGE exception and the Micro-Delta residue path.
  2. Author signal (convention, not server-enforced): the PR-body template gains the optional line Micro-review eligible: <class> — <why>; the reviewer decides, escalation to the full form is always free, the reverse never happens.
  3. Bounded-repair guard (Grace, this ticket): a repair of a named prescription stays micro-eligible only while it touches NO site the prescription did not name — a widened repair is a new change wearing a repair's eligibility and takes the full form. Documented in §6.4 + the template asset.
  4. Template asset: .agents/skills/pr-review/assets/pr-micro-review-template.md, so the live-anchor loader has a source like every other form.

C — Byte compensation (accretion defense)

  1. pull-request-workflow.md (21,989 B) + pr-review-guide.md (33,447 B) together must not grow: the added table + micro-review documentation is paid for by at least equal trims across the two files — the test-evidence diet counts; §6.2 trims to denser wording with the SHAPE preserved (Grace: pointer, never deletion — the shape lives there); further candidates: §1 CLI prose (the tool's --help owns the option surface), guide §7.1/§7.3 compression.

Acceptance Criteria

  • AC-1: The PR-body template documents the ## AC Evidence table with an example of ≤ 10 lines showing BOTH proof forms (CI-covered spec reference; outside-CI command + receipt).
  • AC-2: The lint fails a PR whose close target carries structured ACs when the table's row count misses the ticket's AC count or any row's proof slot is empty; the No structured ACs on #N line passes only against a ticket genuinely lacking a structured AC list.
  • AC-3: CI workflow and agent-preflight enforce identically (one owning implementation; guard-ci-parity green).
  • AC-4: The template codifies the outside-CI-only rule for test mentions; ## Test Evidence collapses to one line when CI carries all coverage.
  • AC-5: Close-target API unreachable → a clearly-caused warning, never a false fail (fail-open on infrastructure, fail-closed on content).
  • AC-6: pr-review-guide.md §6.4 documents the SHIPPED micro-review (form, eligibility semantics incl. the fleet-critical-zone exclusion, the bounded-repair no-unnamed-sites guard, the three-axes disambiguation) and the workflow's body template carries the author signal line — both audiences read the same criteria; the template asset exists.
  • AC-7: The ## AC Evidence count excludes struck-through ticket AC lines (honest in-flight amendments never fire the gate); the outside-CI carve-out names mutation/diagonal results explicitly.
  • AC-8: Net loaded bytes of pull-request-workflow.md + pr-review-guide.md after the change ≤ their sum before (55,436); the trims are named in the PR.

Avoided Traps

  • The lint checks proof presence, never proof truth — false certification stays possible but becomes a machine-readable, spot-auditable artifact; audit remains review/sweep work.
  • The certificate does not duplicate CI: red CI already blocks review; the table covers what CI cannot see (AC mapping + outside-CI evidence).
  • Micro review is a smaller FORM, never a deferral mechanism — findings are still discharged in-place on the same PR (the v13.0 A+FU flood is the recorded anti-pattern).
  • No migration of historical tickets to structured AC format; count-in-order matching covers the dominant anonymous-checkbox format.
  • One AC-extraction implementation shared by CI and preflight; one micro-anchor definition shared by MCP validator and CI lint — no second parser to drift.

Provenance

Operator direction (paired session 2026-08-21/22) following the 55-RA classification sweep; sharpened twice same session: "PRE CI times … don't mention tests at all, unless outside of CI => red CI means no review" and "authors or reviewers MUST know what a micro review is, and when to use it" (measured on PR #17528's full-form review of a +11/−5 bounded repair: 8m18s + delegated audit subagent). Context: PR-creation→approval bottleneck exploration, AC-skip observation, v13.0 A+FU flood history.

Filed by Clio (Claude Fable 5, Claude Code). Origin Session ID: 8947f450-e0c3-424b-8aa1-1e52ea33c03f

Contract Ledger

Surface Signature / grammar Consumer Notes
AC resolver resolveTicketAcs(number, {cwd, execFileSyncImpl, timeoutMs}){state: 'ok'|'missing'|'unknown', acs: String[]} runPrBodyGate (wired unconditionally), specs (injected) missing = verdict failure (content); unknown = warning (transport)
Certificate row | AC-k | proof | or | #N AC-k | proof | parseAcEvidenceRows{id, proof, target|null} qualifier MANDATORY with >1 Resolves; single target owns the shorthand
Id register per-target ids run exactly 1..n in order validatePrBody duplicates/gaps/misorder fail — one proof never discharges two criteria
Empty declaration No structured ACs on #N, N ∈ close targets validatePrBody foreign N fails; rows + declaration for one target contradict (shape-level)
Ticket AC extraction top-level - [ ] / - AC-n: bullets under ## Acceptance Criteria; ~~struck~~ excluded extractTicketAcLines count-in-order contract; no ticket migration
Micro selection real H1 # PR Micro-Review outside fences isMicroReview (service) + CI twin a full review DISCUSSING the form is never rerouted
Micro floor header + **Class:** (micro|contained|mechanical) + **Verdict:** + **Glance:** + - **Origin Session ID:** getMicroReviewTemplateMisses, CI twin, validate_pr_review_body template: reports assets/pr-review-micro-review-template.md (wired in selectedPrReviewTemplatePath)
tobiu referenced in commit 25aa072 - "feat(ai): the PR body certifies every close-target AC; the shipped micro review becomes knowable (#17527) (#17532) on Aug 22, 2026, 3:13 AM
tobiu closed this issue on Aug 22, 2026, 3:13 AM