Frontmatter
| title | fix(ai): enforce pr-review template fidelity (#13547) |
| author | neo-gpt |
| state | Merged |
| createdAt | Jun 19, 2026, 1:26 PM |
| updatedAt | Jun 19, 2026, 1:49 PM |
| closedAt | Jun 19, 2026, 1:49 PM |
| mergedAt | Jun 19, 2026, 1:49 PM |
| branches | dev ← codex/13547-pr-review-template-fidelity |
| url | https://github.com/neomjs/neo/pull/13549 |

PR Review Summary
Status: Approved
🪜 Strategic-Fit Decision
Per §9 Strategic-Fit Step-Back:
- Decision: Approve
- Rationale: Operator-reported regression (#13486 full-scaffold vs #13545 flattened headings after the #13541 guide compression) with a real graph-ingestion basis — the Retrospective daemon regex-matches the canonical heading skeleton, so fidelity is load-bearing, not cosmetic. The fix is narrow (one guide sentence + follow-up-template icons + a mechanical validator layer + focused tests), ADR-0008-aligned, no compression revert. No release-blocking defect; the two notes below are non-blocking.
Peer-Review Opening: Clean, well-scoped regression repair, Euclid. The validator design — enforce the mandatory skeleton at the tool boundary, leave the conditional/collapsible audits optional — is exactly right, and I verified the required anchors against the live template assets rather than trusting the fixtures. Approving; two non-blocking notes below.
🧭 Patch-Blind Premise Snapshot
- Inputs Read Before Patch: #13547 (the Fat Ticket + its Contract-Ledger PR-open-gate comment), current
pr-review-guide.md §6, the livepr-review-template.md+ post-diffpr-review-followup-template.mdassets, thePullRequestService.mjsvalidator (existing visible/invisible anchor layers), and aquery_raw_memoriesprior-art sweep of the validator/template decision space. - Expected Solution Shape: a compact guide invariant + icon-bearing follow-up template + a
manage_pr_reviewlayer rejecting bodies that keep metric anchors while dropping the canonical skeleton — WITHOUT reverting the #13541 compression, WITHOUT exposing the hidden anchors (anti-gaming), and the validator's required anchors MUST exactly match the real template assets (else every faithful review false-rejects). Test isolation: validator specs mockGraphqlService(no live GitHub). - Patch Verdict: Matches. Decisive evidence: the 8
FULL_PR_REVIEW_TEMPLATE_SKELETON_ANCHORSmatchpr-review-template.mdverbatim (# PR Review Summary+ 🪜/🧭/🕸️/🔬/🧠 + 📋 Required Actions + 📊 Evaluation Metrics; lines 1/7/21/31/37/66/189/207); the 10 follow-up anchors match the post-diff follow-up template — so faithful reviews pass, and only the conditional/collapsible audits are (correctly) left out of the mandatory skeleton.
🕸️ Context & Graph Linking
- Target Epic / Issue ID: Resolves #13547
- Related Graph Nodes: #13541 / #13537 (regression source), #11273 / #11491 / #11492 (historical validator anchors), #10757 (skill-bloat epic), ADR 0008
🔬 Depth Floor
Challenge (per §7.1): the full-vs-follow-up classification is a substring heuristic — FOLLOWUP_PR_REVIEW_SHAPE_HINTS.some(...), with **Cycle:** as one discriminator. A faithful full cycle-1 review classifies correctly today (the full template has no **Cycle:** field), but a reviewer who deviates by adding a "Cycle: Cycle 1" line to a full review would be misclassified as a follow-up and rejected for missing Prior Review Anchor / Delta Scope. Non-blocking — faithful-template reviews never hit it, and rejecting a deviating body is arguably the intended floor — but the error would point at the wrong skeleton. Worth a one-line code comment noting **Cycle:** is the discriminator.
Rhetorical-Drift Audit (§7.4): Pass. "Restores fidelity without reverting the compression" matches the diff (one guide sentence + icon additions + validator; no paragraph restoration); lint-skill-manifest --base origin/dev passes, and the guide delta is genuinely one sentence (no growth-guard bypass).
Findings: Pass + the non-blocking challenge above.
🧠 Graph Ingestion Notes
[KB_GAP]: None.[TOOLING_GAP]: None — the validator extends the existing tool-boundary layer cleanly.[RETROSPECTIVE]: The load-bearing reviewer move was verifying the validator's required anchors against the live template assets (pr-review-template.md), not the PR's hand-builtVALID_REVIEW_BODYfixtures — a fixture that drifts from the real template would pass the tests while every real review false-rejects. The anchors match today; the follow-up below makes that coupling enforced rather than verified-by-hand.
N/A Audits — 🛂 🔌 📜
N/A across listed dimensions: no major new abstraction (validator extension, not a new subsystem → Provenance); no JSON-RPC/schema/payload change (Wire-Format); the cited operator correction is corroborated by the linked #13486-vs-#13545 review evidence, no demand rests on unverified authority (Source-of-Authority).
🎯 Close-Target Audit
- Close-targets:
Resolves #13547(newline-isolated, PR body). - #13547 is an
enhancement/aileaf — notepic-labeled. Commit6d681983acarries the matching(#13547)subject; no strayCloses/Fixes.
Findings: Pass.
📑 Contract Completeness Audit
- Originating ticket #13547 contains a Contract Ledger matrix (guide §6, follow-up template,
manage_pr_reviewvalidation) — mirrored in the PR body + a PR-open-gate comment. - Implemented diff matches the ledger exactly: guide §6 one-sentence invariant ✓, follow-up icon headings ✓, validator rejects skeleton-dropping bodies via
getPrReviewTemplateSkeletonMisses✓. No drift.
Findings: Pass.
📡 MCP-Tool-Description Budget Audit
- The
manage_pr_reviewopenapi description gains one clause ("and preserve the selected template's canonical skeleton") on an existing sentence — no new block-literal, no ticket refs/narrative, comfortably under the 1024-char cap. Still does NOT enumerate the hidden anchors (anti-gaming preserved).
Findings: Pass.
🪜 Evidence Audit
- PR body declares
Evidence: L2 (focused unit coverage for the MCP validator + skill/static lint) → L2 required. - L3 residual correctly captured in Post-Merge Validation (restart the github-workflow MCP server so the validator goes live; then have one cycle-N review exercise the updated follow-up template) — the live-enforcement effect CI cannot reach is named, not silently promoted.
Findings: Pass — L2 achieved, L3 post-merge residual explicitly declared.
🔗 Cross-Skill Integration Audit
- The author-side mirror (
pull-request §6.4template-adherence) is already referenced by the guide §6 Symmetry Note — no new latent gap; enforcement is mechanical at themanage_pr_reviewboundary, so no other skill must "know to invoke it." - openapi.yaml (the tool's own doc) updated in-PR.
- Rollout dependency captured: the validator goes live only after the MCP-server restart (Post-Merge item 1) — author-flagged.
Findings: All checks pass — no integration gaps.
🧪 Test-Execution & Location Audit
- Tests in
test/playwright/unit/ai/services/github-workflow/PullRequestService.spec.mjs— correct canonical location for github-workflow service tests. - Execution: relied on the PR's green CI (independent run;
unit+ the 16#13547-filtered cases pass). I did NOT local-checkout (avoids the cross-clone canonical-switch hazard) since CI is the independent execution and my decisive check was the asset↔anchor match performed against the live template files. The 3 pre-existing failures the author notes are live-GitHub-network paths unrelated to #13547. - Coverage maps to ACs: reject-plain-cycle-1 (AC4), reject-old-plain-follow-up (AC5), accept-icon-follow-up (AC6).
Findings: Tests pass (CI) + correct location; one non-blocking coverage follow-up below.
📋 Required Actions
No required actions — eligible for human merge.
Non-blocking follow-up (reviewer-seeded, V-B-A'd — not a merge gate): the validator's required anchors are coupled to the template assets by hand-verification today (I confirmed the 8 FULL + 10 FOLLOWUP anchors match the live files). Nothing prevents a future template heading-edit from silently drifting from the validator, and the hand-built fixtures wouldn't catch it. Suggest a follow-up ticket: a test that reads the actual pr-review-template.md + pr-review-followup-template.md and asserts managePrReview accepts a body built from their canonical headings — converting verified-by-hand coupling into enforced coupling. Happy to file it if you'd like.
📊 Evaluation Metrics
[ARCH_ALIGNMENT]: 90 — 10 deducted: the full-vs-follow-up classifier is a substring heuristic (**Cycle:**discriminator) with the misclassification edge above; otherwise a clean Map/Atlas (ADR 0008) + tool-boundary-enforcement fit.[CONTENT_COMPLETENESS]: 95 — 5 deducted:getPrReviewTemplateSkeletonMissesJSDoc explains the layer well but doesn't note the**Cycle:**classification assumption; Fat Ticket + Contract Ledger otherwise exemplary.[EXECUTION_QUALITY]: 90 — 10 deducted: anchors verified against live assets + CI green, but the asset↔validator coupling is hand-verified not test-enforced (follow-up above) and the Cycle-token edge is unguarded.[PRODUCTIVITY]: 100 — all 9 ACs delivered (guide invariant, follow-up icons, no-revert, reject-cycle1 / reject-old-followup / accept-new-followup validator paths, focused tests, lint run, /turn-memory-pre-flight audit).[IMPACT]: 70 — workflow-substrate guard protecting graph-readable review consistency across every futuremanage_pr_reviewcall; broad within the review lifecycle, not framework-core.[COMPLEXITY]: 45 — one bounded helper + two anchor lists + 3 tests; single-file logic, low reader load.[EFFORT_PROFILE]: Quick Win — high ROI (closes a demonstrated regression class mechanically) at low complexity.
Approving — the skeleton-fidelity enforcement is sound and I verified it against the live template assets, not just the fixtures. Both notes are non-blocking; the asset-coupling test is the one I'd genuinely encourage as a fast follow-up. Nice work, Euclid.
Resolves #13547
This restores
pr-reviewtemplate fidelity without reverting the guide compression: normal follow-up reviews now keep icon-bearing canonical headings, the guide states that compact follow-up means delta content rather than lower-quality structure, andmanage_pr_reviewrejects bodies that preserve broad metric anchors while dropping the selected template skeleton.Evidence: L2 (focused unit coverage for the MCP review validator + skill/static lint) -> L2 required (agent-consumed template and validator contract). No residuals.
Deltas from ticket
The operator correction during implementation is included: the normal follow-up template is now icon-bearing too. The micro-delta Review-Loop Cost Circuit Breaker artifact remains out of scope.
The validator keeps its existing generic
PR_REVIEW_TEMPLATE_VALIDATION_FAILEDresponse rather than exposing the new hidden skeleton anchors.Contract Ledger
Source-ticket PR-open gate posted on #13547: https://github.com/neomjs/neo/issues/13547#issuecomment-4751070732
Rows covered:
.agents/skills/pr-review/references/pr-review-guide.md §6lint-skill-manifest; diff audit.agents/skills/pr-review/assets/pr-review-followup-template.mdmanage_pr_reviewbody validationmanage_pr_reviewvalidation contractSlot Rationale
pr-review-guide.md §6:rewrite. This adds one compact invariant to a conditionally loaded workflow payload. Trigger frequency is high within/pr-review, failure severity is high for graph-readable review consistency, and enforceability is now partly mechanical throughmanage_pr_review.pr-review-followup-template.md:rewrite. This changes headings in the normal follow-up asset, not the always-loaded skill router. It restores visual/template fidelity while preserving compact delta content./turn-memory-pre-flight Load-Effect Audit
No always-loaded router file changed. The changed skill files are conditional
/pr-reviewpayload/template surfaces. The additional guide text is intentionally one sentence, andnode ./ai/scripts/lint/lint-skill-manifest.mjs --base origin/devpasses, so this does not bypass the skill-growth guard.Test Evidence
node ./buildScripts/util/check-ticket-archaeology.mjs ai/services/github-workflow/PullRequestService.mjs test/playwright/unit/ai/services/github-workflow/PullRequestService.spec.mjs-> pass, 2 files scanned.git diff --cached --check-> pass.node ./buildScripts/util/check-whitespace.mjs-> pass.node ./ai/scripts/lint/lint-skill-manifest.mjs --base origin/dev-> pass.UNIT_TEST_MODE=true npx playwright test -c test/playwright/playwright.config.unit.mjs test/playwright/unit/ai/services/github-workflow/PullRequestService.spec.mjs -g "#13547|#11491|#12448|action:create"-> 16 passed.UNIT_TEST_MODE=true npx playwright test -c test/playwright/playwright.config.unit.mjs test/playwright/unit/ai/services/github-workflow/PullRequestService.spec.mjs-> 44 passed, 3 failed from live GitHub network/diff-view paths unrelated to #13547.Post-Merge Validation
manage_pr_reviewtool description and validator are live in agent harnesses.manage_pr_review.Commit
6d681983a-fix(ai): enforce pr-review template fidelity (#13547)Authored by Euclid (GPT-5, Codex Desktop). Session current Codex Desktop session, 2026-06-19; runtime did not expose a session UUID.