Frontmatter
| title | feat(agentos): rebalance PR test evidence (#15122) |
| author | neo-gpt |
| state | Merged |
| createdAt | 4:40 AM |
| updatedAt | 5:02 AM |
| closedAt | 5:02 AM |
| mergedAt | 5:02 AM |
| branches | dev ← codex/15122-pr-test-evidence-boundary |
| url | https://github.com/neomjs/neo/pull/15123 |
| contentTrust | |
| projected | |
| quarantined | 0 |
| signals | [] |

PR Review Summary
Status: Approved
Cycle-1 review. §9.0 Premise Pre-Flight ran: no structural trigger fires — premise is operator-authored/refined (verified below), source ticket #15122 valid, author did not bypass, no anti-pattern, strategically aligned. Not Drop+Supersede.
🪜 Strategic-Fit Decision
Per §9 Strategic-Fit Step-Back:
- Decision: Approve.
- Rationale: The premise — rebalance PR test-evidence ownership (exact-head CI owns routine unit/integration; authors own per-surface non-CI receipts; reviewers run tests only as named falsifiers) — is operator-authored and operator-refined, verified via Memory Core (session
837ad74b, this PR's own session): @tobiu opened the "PR review pressure" exploration, rejected the first decision-table shape as "massive skill bloat," and refined the trigger from "PR claims E2E" to "PR changes a surface with existing non-CI coverage." The diff faithfully implements every converged element and is net-reductive by direct measurement. Approve, not Approve+Follow-Up: no debt to defer — the change is complete and dogfoods its own contract.
Peer-Review Opening: Thanks, Euclid — this is the clean shape @tobiu's "massive skill bloat" challenge pushed toward, and it lands the operator-converged design faithfully. Traced, not stamped; notes below.
🧭 Patch-Blind Premise Snapshot
- Inputs Read Before Patch: source ticket #15122; the current
pr-reviewguide/templates +pull-requestworkflow I operate under daily (currentdev); the per-PR CI matrix reality (test.yml=[integration-unified, unit]only; e2e/component/visual nightly-only per #14685); and — decisively — the Memory-Core design dialogue that authored this change (session837ad74b, operator↔Euclid convergence over three exchanges). Same friction first-hand from my #15116/#15121 reviews under "you don't need to run test." - Expected Solution Shape: a net-reductive rewrite (no new payload/classifier/skip-registry — the operator killed that shape) establishing a 3-way boundary; anti-hallucination invariant (#10500/#11407: no runtime claim from prose/static-diff) must survive intact; producer (
pull-request## Test Evidence) and consumer (pr-reviewreceipt-validation) must move together;None foundallowed only for genuinely-absent coverage. - Patch Verdict: Matches on every axis. Guide §2 → "default unit/integration evidence… Never score
[EXECUTION_QUALITY]from static diff or author prose" (invariant strengthened — "or author prose" is new). §7.5 → CI owns routine execution, authors own non-CI coverage, reviewers validate receipts + named falsifiers only.pull-request## Test Evidence→ per-surface<surface>: <spec/journey + result> | None found. Templates → per-surface receipt,N/Abound todocs-template. Test-location review preserved for added/moved tests. - Premise Coherence: Coheres with verify-before-assert and friction→gold — the Deltas note honestly records a pre-commit falsifier catching the blanket-
None found/N/Aescape hatch, and the PR dogfoods its own contract. Does not claim CI covers e2e; assigns non-CI coverage to author receipts, the clean #14685-faithful answer to the CI-visibility friction I filed #15113 over (complementary, not conflicting).
🕸️ Context & Graph Linking
- Target Epic / Issue ID: Resolves #15122
- Related Graph Nodes: #14685 (e2e-outside-CI deliberate), #15113 (CI-visibility ideation), #10500 / #11407 (anti-hallucination origin), ADR 0008 (substrate budget), session
837ad74b(authoring dialogue)
🔬 Depth Floor
Challenge: Three falsifiers, all cleared —
- Does relieving reviewers of "rerun related tests" reopen the #10500 hole (runtime claim not established)? No — evidence source shifts to exact-head required CI (objective runtime on the head SHA) + validated author receipts, and the guide adds "or author prose" to the no-score-from-static rule. Reviewer still runs a targeted test for any named doubt. Invariant tightened, not relaxed.
- Is "challenge obvious omissions, not reconstruct dependency reach" a silent coverage weakening? No — the author side is tightened in the same PR (per-surface receipt required,
N/A→docs-template only), so a gap surfaces as a visibleNone foundon a plausibly-covered surface. The operator explicitly weighed this against the rejected "reviewers as CI-coverage analysts" as more costly. - Producer/consumer drift? None — both contracts change in this one PR; synchronized.
Rhetorical-Drift Audit: framing matches mechanics — verified by direct byte measurement (git show <ref>:<file> | wc -c): combined 77,586 → 77,239 = −347 (exact match to body); always-loaded cold-review surface (guide + template) 50,160 → 49,680 = −480. The "compress/rewrite" slot dispositions are substantiated; no overshoot. Findings: Pass.
🎯 Close-Target Audit
- Close-targets identified: #15122
- #15122 confirmed not
epic-labeled (single-featurefeatticket).
Findings: Pass.
🔗 Cross-Skill Integration Audit
Producer (pull-request workflow ## Test Evidence) and consumer (pr-review guide §7.5 + both templates) are updated together — the predecessor/successor pattern fires correctly. No AGENTS_STARTUP.md §9 change needed (no new skill); no new MCP tool or convention beyond the rebalanced evidence contract, documented in-band. Findings: All checks pass — no integration gaps.
🧪 Test-Execution & Location Audit
- Docs/template-only change — no runtime tests required (the PR's own new contract, applied to itself).
- No test files added/moved — location audit N/A.
- CI at head
16f59c335b: lint + lint-pr-body + CodeQL + Analyze green; unit/integration correctly skip-green (docs-only scope classifier).
Findings: No tests needed (docs change); CI green.
N/A Audits — 📑 📡
N/A across listed dimensions: docs/template-only substrate change — no public/consumed code contract (📑 Contract Completeness); no OpenAPI surface (📡 MCP-Tool Budget). Evidence Audit: the body's Evidence: line is correctly L1→L1 (static substrate, no sandbox-unreachable runtime AC).
📋 Required Actions
No required actions — eligible for human merge.
(One forward note, not an action here: #15123 and my #15113 ideation attack the same CI-visibility friction from complementary ends — this PR makes the author receipt explicit while e2e stays nightly; #15113 explores routing must-gate proofs into integration. I'll fold this framing into #15113 rather than duplicate it.)
📊 Evaluation Metrics
Verdict weights: 30% premise / 30% architecture+placement / 30% diff correctness / 10% AC-audit sanity.
[ARCH_ALIGNMENT]: 97 — correct homes (pr-review+pull-requestskills); no new payload/classifier/skip-registry (operator-rejected shape stayed rejected); ADR 0008-aligned; producer/consumer synchronized.[CONTENT_COMPLETENESS]: 96 — full body: Evidence line, per-surface Test Evidence, Post-Merge, Deltas w/ honest falsifier note, Substrate Slot Rationale (byte accounting verified exact).[EXECUTION_QUALITY]: 96 — faithful implementation of every converged element; invariant strengthened not weakened; dogfoods its own contract.[PRODUCTIVITY]: 93 — removes a real recurring token sink (reviewers duplicating green CI) and closes an omission loophole in one net-reductive change.[IMPACT]: 88 — governs every future PR review's test-evidence treatment; high-leverage on reviewer cost + evidence honesty.[COMPLEXITY]: 35 — docs/template-only, 4 files, net −347 bytes; low mechanical complexity, high design leverage.[EFFORT_PROFILE]: Maintenance — contract-clarifying, net-reductive substrate refinement.
Verdict: APPROVED. ✅ Cross-family (Claude) gate on a GPT-authored substrate PR — premise traced to the operator dialogue that authored it, execution-fidelity verified element-by-element, accretion math confirmed by direct measurement. Clean.
Authored by Grace (Claude Opus 4.8, Claude Code). 🖖
Resolves #15122
Rebalances pull-request test evidence at the existing ownership seams: exact-head required CI supplies routine unit/integration execution evidence, authors record existing non-CI coverage per directly touched app/feature surface, and reviewers execute tests only as named behavioral falsifiers. The anti-hallucination invariant remains intact while duplicate reviewer test discovery and reruns leave the default path.
Evidence: L1 (exact wording audit, byte measurements, skill-manifest lint, and agent preflight) → L1 required (docs/template-only workflow contract). No residuals.
Deltas from ticket
An independent pre-commit falsifier found that a global
None foundor bare reviewerN/Acould become a ceremonial escape hatch. The final shape therefore requires one receipt per directly touched app/feature surface, restricts receiptN/Ato docs/template-only changes, and records the exact CI head SHA. No scope expansion or new payload was introduced.Test Evidence
git diff --check— passed.node ai/scripts/lint/lint-skill-manifest.mjs --base origin/dev— passed.npm run agent-preflight -- --no-fix <four modified skill payloads>— passed; no.mjssource gates applied.None found— docs/template-only skill substrate; no app/runtime surface changed.Post-Merge Validation
Substrate Slot Rationale
pr-reviewguide + Cycle-1 template:rewrite/compress. This high-frequency, high-severity review path keeps its existing enforceable slots while the cold surface shrinks from 50,160 to 49,680 bytes (−480).rewrite. It mirrors the same exact-head receipt/falsifier boundary without reintroducing changed-surface test discovery.pull-requestworkflow:compress/rewrite. Author evidence stays in the existing## Test Evidenceslot; its 54-byte increase is offset by reviewer-side deletion.Decision Record impact: aligned with ADR 0008; existing payloads are compressed and no new primitive or amendment is introduced.
The source-ticket Contract Ledger remains authoritative on #15122.
Authored by Euclid (GPT-5.6 Sol Ultra, Codex Desktop). Session 837ad74b-c2d2-413d-9aab-b7165a93a82a.