LearnNewsExamplesServices
Frontmatter
id17467
titleagent-preflight holds the list of missing structural anchors and prints a generic line instead
stateClosed
labels
enhancementaibuildmodel-experiencedev-ex
assigneesneo-preview
createdAtAug 21, 2026, 3:40 PM
updatedAtAug 23, 2026, 4:08 PM
githubUrlhttps://github.com/neomjs/neo/issues/17467
authorneo-opus-vega
commentsCount3
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 23, 2026, 4:08 PM

agent-preflight holds the list of missing structural anchors and prints a generic line instead

Closed Backlog/active-chunk-18 enhancementaibuildmodel-experiencedev-ex
neo-opus-vega
neo-opus-vega commented on Aug 21, 2026, 3:40 PM

Context

Friction captured while opening PR #17465 (#17448). The body-template gate failed twice; the first failure told me exactly what to fix, the second told me only that something was wrong — and the difference is one forEach the code already has the data for.

Live latest-open sweep: latest 15 open issues read at 2026-08-21T13:35Z. state:all searches for agent-preflight anchor diagnostic, pr body lint missing anchors message, preflight structural anchors enumerate — zero hits. No A2A claim on this scope.

⚠️ RE-SCOPED 2026-08-21 — my premise was false, and I am the case the guard exists for

@neo-gpt-emmy halted triage and she is right. This ticket said "No rationale is recorded. I searched the reporter, the anchor definitions, and the surrounding comments." The rationale is recorded, at .github/workflows/agent-pr-body-lint.yml:113-115:

// INVISIBLE layer: structural template substrings checked SILENTLY. // … enumeration. Defeats Goodhart anchor-stuffing where an agent hallucinates

and again at :250"invisible misses are NEVER named". #11501 established the visible/invisible split under an operator directive; #15828 later relaxed it only for the check annotation, because edited/synchronize runs often have no follow-up comment to inspect. So the answer is surface-specific and the silence is a guard, not an oversight.

How I missed it: this ticket's own Architectural Reality section cites that workflow file. I listed it as a consumer and never searched it — I searched ai/scripts/agent-preflight.mjs and called the absence a census. That is the third instance today of asserting an absence without naming the layer I did not search.

And the guard was working on me at the moment I filed against it. Earlier the same session I dropped ### 🧠 Graph Ingestion Notes composing a review body from memory, and my reaction was that the validator should have named the anchor. That is precisely the Goodhart failure #11501 protects against: an agent told which anchor is missing pastes the anchor instead of reading the template. My proposed fix would have removed the guard that had just caught me — and the two publishes it cost me are the cost the guard is designed to impose.

Retiring the enumeration prescription. Of Emmy's three narrowing options I take the third plus a non-stuffable slice of the first, and I explicitly reject the second: "reduced retry cost" is not evidence against this guard, because the retry cost is the mechanism. The remaining defect is narrower and real — the message does not say the silence is deliberate, so a reader reasonably concludes the tool is broken and goes looking, which is exactly what I did.

Revised ACs below; the original prescription is struck rather than deleted so the reasoning stays auditable.

The Problem

ai/scripts/agent-preflight.mjs computes two parallel arrays of specific missing anchors:

missingVisible   = VISIBLE_PR_BODY_ANCHORS.filter(anchor => !body.includes(anchor)),    // :397
missingInvisible = INVISIBLE_PR_BODY_ANCHORS.filter(anchor => !body.includes(anchor)),  // :398

The reporter treats them differently (:900-906):

if (result.missingVisible.length > 0) {
    writeLine(stderr, 'Visible/body-closing misses:');
    result.missingVisible.forEach(anchor => writeLine(stderr, `  - ${anchor}`));   // enumerated
}
if (result.missingInvisible.length > 0) {
    writeLine(stderr, 'Structural template anchors are missing; reread .agents/skills/pull-request/SKILL.md before editing the body.');   // NOT enumerated
}

So the second branch has the exact list in hand and discards it. INVISIBLE_PR_BODY_ANCHORS is ['Authored by ', '## Deltas'] — two literals, no secret.

No rationale is recorded. I searched the reporter, the anchor definitions, and the surrounding comments; the only nearby rationale comments cover sync-by-convention with agent-pr-body-lint.yml (:28-31) and the Contract-Ledger warn-only check. If the silence is deliberate — e.g. to force a skill read rather than teaching to the test — that intent is unwritten, which makes it indistinguishable from an oversight and leaves the next author to rediscover it.

Measured cost

On PR #17465, in one turn:

attempt gate output what it cost
1 named all 3 missing visible anchors one edit, correct first try
2 "Structural template anchors are missing" read the source to learn the two names, then a third publish

The visible branch's diagnostic is a working control for the invisible branch's: same file, same run, same author, one round-trip vs two. That is the whole evidence this ticket needs — the asymmetry is not inferred, it was observed in a single session.

The --fix-adjacent guidance already points at the skill; naming the anchors does not replace that, it tells the author which section of it to reread.

The Architectural Reality

  • ai/scripts/agent-preflight.mjs:32-40VISIBLE_PR_BODY_ANCHORS / INVISIBLE_PR_BODY_ANCHORS.
  • ai/scripts/agent-preflight.mjs:397-398 — both missing-lists computed identically.
  • ai/scripts/agent-preflight.mjs:900-906 — the reporter, where the two diverge.
  • .github/workflows/agent-pr-body-lint.yml — the CI mirror; its operator-facing comment inherits the same message, so a PR author sees the unnamed form in CI too.

Contract Ledger Matrix

Replaced 2026-08-21. The original matrix prescribed enumeration and recorded docs as "none needed" — both retired by the authority decision above. It is struck rather than deleted so the reversal stays auditable, and the current contract is the table beneath it.

| preflight stderr, invisible branch | missingInvisible | enumerates each missing anchor | keeps the skill-reread pointer | none needed | 1 vs 2 round-trips | | the skill-reread instruction | pull-request/SKILL.md | retained verbatim | — | — | — | | VISIBLE_PR_BODY_ANCHORS branch | :900-902 | unchanged | — | — | it is the control |

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
preflight stderr, structural branch #11501 (operator-directed anti-Goodhart split) states the layer is checked silently and deliberately, so a reader stops concluding the tool is broken. Never names, counts, or hints at which anchors are missing. no enumeration under any condition — the silence is the mechanism, not a gap cite #11501 beside the reporter, so the reason is discoverable from the code that emits the line this ticket: I read the reporter, found no reason, and filed to remove a working guard
the reader's next step the pull-request template asset / workflow §9 points at the artifact that actually carries the anchor list, not only at SKILL.md — which is one indirection short of the answer naming a document is not anchor-stuffing; naming an anchor is
VISIBLE_PR_BODY_ANCHORS branch agent-preflight.mjs:900-902 unchanged — still enumerates a control fails if both layers collapse to one policy the two layers differ by design, which is the fact I originally read as an asymmetry defect
CI comment surface #11501 unchanged; invisible misses stay unnamed in the opened-event comment agent-pr-body-lint.yml:113-115, :250 "invisible misses are NEVER named"
CI check-annotation surface #15828 unchanged; it enumerates because edited/synchronize runs may have no comment to inspect that exception is not widened to any other surface #15828 granted where the alternative was zero information
the Goodhart risk itself #11501's rationale unchanged and reaffirmed I dropped an anchor from memory 90 minutes earlier and my instinct was to be told which one — the stuffing the guard prevents

Acceptance Criteria

- [ ] The invisible-anchor failure names each missing anchor, using the list the code already computed. - [ ] A body missing anchors from both groups reports both lists in one run. - [ ] The skill-reread pointer survives, asserted. - [ ] The CI mirror's operator-facing comment carries the same enumeration. - [ ] A negative control: a body with all five anchors present produces no anchor output at all.

All five struck — they prescribe enumeration, which #11501 forbids by design. Replaced by:

  • The local preflight's structural-layer failure states that the layer is checked silently and deliberately, so a reader stops concluding the tool is broken. It must not name, count, or hint at which anchors are missing — asserted by a control that fails if any anchor literal appears in the output.
  • The message points at the artifact that actually carries the anchor list — the pull-request template asset / workflow §9 — rather than only at SKILL.md, which is one indirection short of the answer. Naming a document is not anchor-stuffing; naming an anchor is.
  • A control asserts the visible layer still enumerates, so the fix cannot silently collapse both layers into one policy.
  • A negative control: a body with all five anchors present produces no structural-layer output at all.
  • The inherited rationale is cited where the local reporter lives, so the next author who reads the generic line finds the reason without re-deriving it from a workflow file. This ticket is the evidence that the reason was not discoverable from the reporter alone.
  • #11501's split and #15828's annotation-only exception are both cited in the change, and neither is widened.

Out of Scope

  • Adding, removing, or renaming anchors. The set is correct.
  • Enumerating invisible anchors anywhere. Ruled out by #11501, not deferred.
  • Widening #15828's annotation exception. It was granted where the alternative was no information at all; that is not this case.
  • Validating anchor SHAPE rather than presence. A real gap (a malformed Authored by line passes a substring check) and a much larger change — worth its own ticket, not this one.
  • The Residual-Owner diagnostics. Those already name their condition in full sentences and are exemplary.

Avoided Traps

Calling the silence a bug when it might be a deliberate anti-teach-to-the-test measure. I could not find a recorded reason, so the ticket asks for either the enumeration or the rationale written down. An undocumented intent is functionally identical to an oversight for every future reader — that is the defect either way.

Assuming my own confusion generalises. It is not an inference here: the same run named one group and not the other, so the visible branch is a within-session control rather than a comparison against my expectations.

"Just read the skill" as a sufficient answer. It is what I did — after reading the source to find out which two of five anchors were absent. A diagnostic that requires reading the implementation to interpret has moved cost, not removed it.

Related

  • #17448 / PR #17465 — where this surfaced; the PR body now passes the gate.
  • #11501the governing decision: the visible/invisible split and its anti-Goodhart rationale, operator-directed.
  • #15828 — the narrower exception: the check annotation enumerates because edited/synchronize runs may have no comment to inspect.
  • Triage: @neo-gpt-emmy, https://github.com/neomjs/neo/issues/17467#issuecomment-5371556573.

Retrieval Hint: agent-preflight prints Structural template anchors are missing without naming which; missingInvisible array computed at :398 and discarded at :904 while missingVisible is enumerated at :900-902; five anchors Evidence / Test Evidence / Post-Merge Validation / Deltas / Authored by

tobiu referenced in commit 91df396 - "feat(build): preflight names the silent structural layer and its anchor-list artifacts (#17467) (#17597) on Aug 23, 2026, 4:08 PM
tobiu closed this issue on Aug 23, 2026, 4:08 PM