LearnNewsExamplesServices
Frontmatter
id15828
titlePR-body lint cites a follow-up comment it only posts on `opened` — every edit fails with no diagnostic
stateClosed
labels
bugai
assigneesneo-opus-ada
createdAtJul 24, 2026, 7:46 PM
updatedAtJul 24, 2026, 10:01 PM
githubUrlhttps://github.com/neomjs/neo/issues/15828
authorneo-opus-ada
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJul 24, 2026, 10:01 PM

PR-body lint cites a follow-up comment it only posts on opened — every edit fails with no diagnostic

neo-opus-ada
neo-opus-ada commented on Jul 24, 2026, 7:46 PM

Context

Three instances in one day, across two agents and two model families, all on the same anchor — and each one cost the author a trip into the workflow source to learn which anchor was missing.

# who edit result
1 @neo-opus-ada (earlier) ## Post-Merge Validation → lowercase lint FAILED
2 @neo-opus-ada, PR #15816 ## Residual / Post-Merge Validation (prefix, while folding a review finding) lint FAILED
3 @neo-gpt, PR #15816 ## Review Closure (bounded reviewer polish) lint FAILED

Instance 2 happened with the rule already banked, because the note was written around instance 1's casing and a prefix did not read as the same rule. Instance 3 was a reviewer's own polish on someone else's PR. Different people, different edits, same silent break.

The Problem

.github/workflows/agent-pr-body-lint.yml runs on four event types but posts its diagnostic on exactly one:

on:
  pull_request:
    types: [opened, edited, synchronize, ready_for_review]
...
if (context.payload.action === "opened") { await github.rest.issues.createComment({  }) }

while the failure itself always says:

core.setFailed(`Agent PR body missing required template anchors. See follow-up comment on PR #${pr.number}.`)

So on edited and synchronize the check fails pointing at a comment that was never posted. That is precisely when the diagnostic is most needed — an anchor breaks because someone edited the body, which is an edited event by construction. The opened path, where the comment does exist, is the one case where the author has the template freshest in mind.

The anchors are matched by plain body.includes(...), so any edit to a heading breaks it — casing, prefixing, suffixing, punctuation. The words can all survive and the substring still not.

This is the class the swarm spent today on: the annotation answers "did the lint fail?" when the reader's question is "which anchor?", and it confidently cites evidence that does not exist. An instrument pointing at a receipt it never wrote.

The Architectural Reality

  • .github/workflows/agent-pr-body-lint.ymlVISIBLE_PR_BODY_ANCHORS (Evidence:, ## Test Evidence, ## Post-Merge Validation) and INVISIBLE_PR_BODY_ANCHORS (Authored by , ## Deltas), plus the Resolves #N regex and the stacked-PR ticket guard.
  • The missing-anchor lists are already computedmissingVisible / missingInvisible are in scope at the failure site. The information exists; it just never reaches the annotation.
  • Consumer: every agent-authored PR. The GitHub Checks annotation is what an agent reads first, and often all it reads.

The Fix

Put the answer in the annotation, rather than only in a comment that may not exist:

  1. core.setFailed names the missing anchors directly — the arrays are already in scope, so this is a message change, not new logic.
  2. Keep the rich follow-up comment on opened (it carries the skill-reading guidance and the collapsed detail block, which is worth more room than an annotation gives).
  3. Do not start commenting on every synchronize — that would spam a PR on each push. The annotation is the right surface for the recurring case.

Contract Ledger

Target surface Source of authority Proposed behavior Fallback Docs Evidence
lint-pr-body failure annotation .github/workflows/agent-pr-body-lint.yml Names every missing anchor inline; no longer cites a comment that may not exist none — strictly adds information to an existing failure workflow comment a body edited to break an anchor fails with that anchor named in the annotation
opened follow-up comment same unchanged n/a n/a still posted on opened

Acceptance Criteria

  • A PR body edited to break an anchor fails with the missing anchor named in the check annotation, on edited and synchronize as well as opened.
  • The annotation no longer references a follow-up comment in runs where none is posted.
  • The opened follow-up comment is unchanged.
  • No new comment is posted on synchronize.

Out of Scope

  • Changing the anchor set, the Resolves #N rule, or the stacked-PR guard.
  • Relaxing substring matching to something fuzzier. The anchors are a contract and should stay literal; the defect is that breaking them is undiagnosable, not that they are strict.

Avoided Traps

Making the matcher lenient (trim/casefold/prefix-tolerant) so "better prose" stops breaking it. That trades a diagnosable failure for a silent contract erosion, and the anchors exist so downstream graph ingestion can rely on them. Strict matching is correct; an undiagnosable failure is not.

Commenting on every event. Three instances came from edits, and edit-heavy PRs are normal — a comment per synchronize would be noise that gets muted, which is how a real signal stops being read.

Related

  • PR #15816 — instances 2 and 3, both visible only because @neo-gpt stated plainly that his polish caused one.
  • The banked reference note on the five literal anchors, sharpened today from the casing form to the general form after instance 2.

Decision Record impact: none. ADR-0019 gate: N/A — CI workflow, no ai/ config surface.

Release classification: not release-blocking; agent-workflow ergonomics with a measured three-instance cost.

Live latest-open sweep: checked latest 15 + a targeted search at 2026-07-24T17:47Z; no equivalent found. A2A in-flight sweep: no competing claim — I surfaced this to AGENT:* at 16:56Z offering it, stated I would take it when my own queue cleared, and it has (#15816 and #15819 both merged; #15827 green and seated).

Origin Session ID: e8b8a230-b55f-4d39-acb2-8680bc922399

Retrieval Hint: query_raw_memories("agent-pr-body-lint missing anchor annotation follow-up comment opened only")

tobiu referenced in commit 276c7a5 - "fix(ci): the lint annotation must carry the answer, not cite a missing comment (#15828) (#15829) on Jul 24, 2026, 10:01 PM
tobiu closed this issue on Jul 24, 2026, 10:01 PM