LearnNewsExamplesServices
Frontmatter
id15309
titleReject caller-authored review-budget provenance on create
stateClosed
labels
bugaitestingmodel-experience
assigneesneo-gpt
createdAtJul 16, 2026, 11:05 PM
updatedAtJul 16, 2026, 11:33 PM
githubUrlhttps://github.com/neomjs/neo/issues/15309
authorneo-gpt
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJul 16, 2026, 11:33 PM

Reject caller-authored review-budget provenance on create

Closed Backlog/active-chunk-6 bugaitestingmodel-experience
neo-gpt
neo-gpt commented on Jul 16, 2026, 11:05 PM

Context

PR #15307 merged the review-cost circuit breaker and made submitted review-budget audit tails immutable on UPDATE. Its terminal review exposed the complementary CREATE admission gap: manage_pr_review accepts caller-authored machine provenance before the service has a chance to append its own canonical receipt.

This ticket is the narrow post-merge hardening leaf. It does not absorb the byte-budget guard in #15308.

Freshness / duplicate evidence: live latest-open sweep checked the latest 20 open issues at 2026-07-16T21:04:34Z; no equivalent ticket exists. Recent all-state A2A claim sweep checked 30 messages; Emmy's explicit post-merge handoff is the only overlapping claim, and assigns this lane here.

The Problem

At origin/dev commit b906ec8d27:

  • getReviewBudgetAuditSnapshot() recognizes exact [review-budget-managed] / [review-budget-override] lines and the reserved audit-field grammar in ai/services/github-workflow/PullRequestService.mjs:902-931.
  • CREATE validates only the review template at PullRequestService.mjs:1715-1719, then enters the create path. It never applies the audit-provenance parser before the first GraphQL request at lines 1764-1770.
  • Post-cutover REQUEST_CHANGES later appends a canonical managed/override tail at lines 1825-1842. A caller can therefore pre-seed an exact marker or reserved audit field and produce duplicate, displaced, or falsely attributed provenance.
  • APPROVED and COMMENT creates can persist caller-authored managed markers unchanged because only REQUEST_CHANGES enters the budget appender.
  • The downstream workflow currently treats marker presence as managed provenance via .github/workflows/agent-pr-review-body-lint.yml:166-176; it is telemetry after submission, not an origin authenticator.

The machine-owned receipt is therefore immutable after submission but not reserved at admission.

The Architectural Reality

The owning substrate is the existing GitHub Workflow service:

  • Runtime owner: ai/services/github-workflow/PullRequestService.mjs
  • Consumed MCP contract: ai/mcp/server/github-workflow/openapi.yaml, operation manage_pr_review
  • Existing behavioral owner tests: test/playwright/unit/ai/services/github-workflow/PullRequestService.spec.mjs
  • Downstream observer: .github/workflows/agent-pr-review-body-lint.yml

The Agent OS structure map identifies the existing service and test sibling; this work introduces no new module or directory. Admission belongs in PullRequestService.managePrReview(), before GitHub lookup or mutation. The workflow remains a post-submit bypass detector and must not become the source of provenance authenticity.

Consumer: agent callers of manage_pr_review, plus downstream review-budget telemetry that relies on service-owned markers.

Service boundary: the MCP service owns provenance issuance; callers may supply human-authored review content, never service audit markers or reserved audit fields.

The Fix

  • Add a CREATE-side reserved-provenance validation step before any GraphqlService.query().
  • Reuse/refactor the existing review-budget audit parser so CREATE and UPDATE share one exact grammar while preserving current UPDATE immutability behavior.
  • Reject caller-authored exact managed/override marker lines and reserved audit-field list entries for all CREATE states.
  • Return one stable structured error (proposed code: PR_REVIEW_BUDGET_AUDIT_RESERVED) naming the rejected marker/field classes without mutating GitHub.
  • Preserve the service-owned REQUEST_CHANGES path: after admission passes, it may append exactly one canonical managed/override audit tail.
  • Document the reserved CREATE input contract in the manage_pr_review OpenAPI description.
  • Add adversarial tests to the existing PullRequestService.spec.mjs suite.

Contract Ledger

Target Surface Source of Authority Proposed Behavior Fallback / Edge Case Docs Evidence
Existing PullRequestService.managePrReview({action: 'create'}) PullRequestService.mjs:1690-1877; review-budget contract from #15257 and PR #15307 Reject caller-authored reserved review-budget provenance before any GitHub read/write; service remains the only issuer Return PR_REVIEW_BUDGET_AUDIT_RESERVED with rejected marker/field classes and zero GraphQL calls JSDoc + OpenAPI Unit matrix across APPROVED, COMMENT, and REQUEST_CHANGES
Existing review-budget audit grammar REVIEW_BUDGET_MANAGED_MARKER, REVIEW_BUDGET_OVERRIDE_MARKER, REVIEW_BUDGET_AUDIT_FIELDS, and getReviewBudgetAuditSnapshot() in PullRequestService.mjs:24-36, 902-931 One parser recognizes exact machine-owned markers and reserved field entries for both CREATE admission and UPDATE immutability Incidental prose/substrings that are not exact marker lines or reserved list-field entries remain human content JSDoc Adversarial exact-line vs incidental-prose tests
Existing manage_pr_review OpenAPI operation ai/mcp/server/github-workflow/openapi.yaml:804-868 Tell callers that CREATE bodies must not contain service-owned review-budget provenance Existing argument/state validation remains unchanged OpenAPI OpenAPI/tool registration tests plus focused service suite
Existing downstream workflow telemetry .github/workflows/agent-pr-review-body-lint.yml:166-176 Continues observing service-issued markers and direct-bypass disclosure after submission Direct gh/UI writes remain outside MCP admission No change Existing workflow tests stay green

Decision Record impact

none — this enforces the already-landed review-budget ownership boundary; it does not introduce or amend an architecture decision.

Acceptance Criteria

  • CREATE rejects an exact [review-budget-managed] marker, exact [review-budget-override] marker, duplicate/displaced marker blocks, and every reserved audit-field list entry supplied without a service-owned tail.
  • The rejection applies to APPROVED, COMMENT, and REQUEST_CHANGES creates and occurs before any GraphqlService.query().
  • The structured failure exposes a stable error code plus the rejected marker/field classes.
  • Incidental prose mentioning marker text or words such as “outcome” does not false-positive unless it matches the reserved machine grammar.
  • A valid post-cutover REQUEST_CHANGES create still appends exactly one canonical managed tail; a valid override still appends one override block followed by one managed block.
  • Existing UPDATE immutability behavior remains green.
  • The manage_pr_review OpenAPI description documents the reserved CREATE provenance contract.
  • The focused PullRequestService.spec.mjs suite passes with red-then-green adversarial coverage and zero-call assertions.

Out of Scope

  • The two-file review-skill byte guard owned by #15308.
  • Changing the two-ordinary-RC budget, Drop+Supersede semantics, or override policy.
  • Reworking downstream workflow telemetry.
  • Authenticating or blocking direct gh/UI review submission.
  • Broad review-body sanitization or escaping.

Avoided Traps

  • Workflow-only detection: too late; the false provenance is already persisted.
  • Substring banning: would reject legitimate review prose. Admission uses the exact machine grammar.
  • A second CREATE-only parser: invites drift from UPDATE immutability. The parser grammar is shared.
  • Scope fusion with #15308: byte-budget enforcement is an independent CI concern and already has an owner.

Related

Related: #15257 · PR #15307 · #15308

Origin Session ID: 1200dbd1-5159-45be-84c3-b7e323ad4979

Retrieval Hint: "manage_pr_review create reserved review-budget provenance duplicate managed tail"

tobiu referenced in commit 5738546 - "fix(ai): reserve review-budget provenance on create (#15309) (#15311)" on Jul 16, 2026, 11:33 PM
tobiu closed this issue on Jul 16, 2026, 11:33 PM