The PR-review validator passes a body containing a client name

⛔ CLOSED by operator Drop+Supersede. The acceptance criteria below are NOT live scope. My AC-3 re-decision was edited in at 09:33Z, after the ~09:23Z close — it records what a proportionate shape would have been (~40 lines inside validate_pr_review_body, the surface that actually leaked), not work that is pending. @neo-opus-grace declined to reopen and she is right: two agents both liking a finding is the convergence trap, not authority. A fourth recurrence would be new evidence with a rate to point at — which this never had.

Refs #16520 · found by an operator catch on a review body I published minutes earlier.

Context

validate_pr_review_body exists to gate a review body before it is posted, and agents are required to call it. On 2026-08-05 it returned valid: true for a body containing a client name in prose. The body was published, caught by the operator, redacted, and verified clean — but the validator was in the path and had nothing to say.

Confidentiality is a §critical_gates invariant (rule 9, no client names in public-facing artifacts). It is currently enforced by discipline alone at the review surface.

Why this is severe — the stake is legal, not merely competitive

Operator, 2026-08-05, escalating this ticket to be driven before other lanes:

it can affect clients own reputation, if their clients google for their name and find public bugfix PRs. a reason for us to get sued legally.

That is a different harm model from the one the confidentiality runbook currently records (competitive sensitivity plus contractual confidentiality). The exposure is transitive: a client's own customers search the client name, reach a public bugfix PR in this repo, and read our defect as the client's. The damaged party is not us and not the client's counterparty — it is the client's standing with their market, which is what makes it actionable against us.

Three consequences for this ticket's design:

1. Scope widens to every public authoring surface, not just review bodies and PR bodies. Issue bodies, issue comments, Discussion bodies and Discussion comments are equally googleable and are where the prior two recurrences landed.
2. Fail-closed is non-negotiable, not a nicety. An unchecked body returning a bare pass is indistinguishable from a clean one, and under this harm model the cost of a false pass is legal rather than editorial.
3. A negative predicate is not required for harm. A neutral mention is enough — the searcher supplies the context. The 2026-08-01 specimen leaked an engagement through a config-entry prefix with no prose and no defect attached.

The Problem

The failure is a generator gap, not an author gap, and the repo already knows the difference. The confidentiality runbook's own keystone rule, written after a 2026-06-21 swarm-wide leak, is:

a one-time scrub REGRESSES within the hour without a write-time redaction guard — fix the GENERATOR (denylist at content-write), not just the output.

That lesson was applied to the data-sync pipeline and never to the publish path. Three recurrences of the same mechanism (2026-06-21, 2026-08-01, 2026-08-05) across three different agents' artifacts is the recurrence evidence; the third one passed through a mandatory validator that checks template structure and nothing else.

Why the author-side discipline keeps failing. The leaking token is not decoration — it is load-bearing in the private rationale that motivated the change, so it travels with the reasoning into the public artifact. Generic framing carries identical force ("during an active deployment incident"), which is exactly why the substitution is safe to automate and easy to forget.

The Fix

Not prescribed in detail — one deliberate open question below. The shape:

1. A denylist check inside validate_pr_review_body, failing closed with the matched token and its offset so the author can scrub before posting rather than after.
2. The same check on the agent PR-body lint, since PR bodies are the same surface class and already have a lint job.
3. The denylist is itself confidential and cannot be committed to this public repo in plaintext — see OQ1.

Open Question

OQ1 — where does the denylist live? It cannot be a plaintext array in a public file, which would publish the exact list it protects. Candidate shapes, none chosen here: an env-supplied list (NEO_CONFIDENTIAL_DENYLIST) read at validate time with the check skipped-and-reported when unset; hashed tokens committed and compared against normalised candidates; or the list held in the private business repo and loaded only when that checkout is present. Each trades coverage against reachability differently, and a check that silently no-ops when the list is missing is worse than none because it reads as enforcement. Whoever takes this should settle it before implementing.

Acceptance Criteria

• A review body containing a denylisted token fails validate_pr_review_body, naming the matched token and offset.
• The check fails closed: when the denylist source is unavailable, the validator reports "unchecked" explicitly rather than returning a bare pass, so absence of enforcement is never indistinguishable from a clean body.
• The check lives inside validate_pr_review_body — a call that is already mandatory, on the surface where the leak actually happened (#16520 was a PR review body). One surface, not a dispatch guard.
• Scope is bounded by the observed failure, not by the severity of the harm. Three events, all on artifacts an agent authored through a reviewed path. Surfaces that have never leaked get no permanent guard on every write.
• The Actions-side surfaces (agent PR-body lint, PR-review-body lint) are tracked in #16660 and are not in scope here. They run outside MCP, and their disclosure contract is inverted — a world-readable log means the matched token must NOT be printed, the opposite of the local rule.
• The uncovered path is named by its mechanism. A PR body written with gh pr create --body-file reaches no MCP tool and no validator; that is the path agents use today, including every PR I opened on 2026-08-08. A reader must be able to tell which surface is open without inferring it from a tool name.
• The denylist is not readable from this public repo in plaintext (OQ1 resolved and its rationale recorded).
• A spec asserts both directions: a body with a token fails, and a clean body passes — the second is what stops the check from being satisfied by rejecting everything.
• Case- and separator-insensitive matching, since a config-entry prefix leaked an engagement in the 2026-08-01 specimen without any prose mention.

Out of Scope

• Historical remediation. Pre-existing occurrences across live issues and the resources/content/ mirrors are a known, operator-directed coordinated sweep, not this ticket.
• The revision-history residue. GitHub keeps edit history behind the "edited" dropdown; purge is UI-only and the operator has calibrated body-scrub as the required floor with the purge optional.
• Widening the denylist beyond client/partner names into business-strategy phrasing — that is a judgement call a token match cannot make.

Avoided Traps

• Adding another checklist line to the runbook. The rule and a pre-publish grep step are both already written there, and the mechanism has recurred three times regardless. Per ADR 0007 §5.4, strengthening the gate already in the path beats new surface area.
• Committing the denylist to satisfy the AC. That would publish the list of names the check exists to protect — the failure mode wearing a fix costume.
• A pass-when-unconfigured default. An unchecked body returning valid: true reproduces exactly today's defect while looking enforced.

Related

• #16520 — the review body where this surfaced; redacted, 0 live occurrences verified, 18 sibling artifacts swept clean.

Live latest-open sweep: latest open issues checked plus a targeted search for validator/denylist scope; no equivalent ticket. No [lane-claim] on confidentiality tooling.

Origin Session ID: 11695cce-9854-4be2-80c3-8ea4322298bf

Retrieval Hint: query_raw_memories("validate_pr_review_body passes client name denylist publish path generator guard confidentiality gate")