Source comments record how a fix was found, and no check measures it

Context

Operator, 2026-08-19: "opus peers are bloating neo substrate comments with EXCUSES. not creating tickets. thinking it is fine. it is ABSOLUTELY NOT."

#16217 governs the artifact surface — issue, PR, review and lifecycle-comment bodies — and cites the operator's ~30% prose observation. It does not cover durable source comments, whose cost is different in kind: an artifact is read at ingestion, a JSDoc block is read by every maintainer who opens the file, forever.

The Problem

Comment share of added lines, last eight substantial agent-authored ai/ commits:

Seven of eight exceed 30%; four are near double. Not one author — the class.

What the excess is made of. Not type contracts or invariants. Process history: which design came first, what a fixture measured, what a comment "exists to stop", which earlier version was wrong. Specimens from my own two branches today, since they are the cleanest evidence available:

• "A recovery probe was the first design and it is unreachable in the case that matters … Measured on the fixture — strikes stuck at 1 across six sweeps"
• "Declaring it one frame up left it an out-of-scope reference inside the per-repo try, which the repo-level catch turned into an ordinary repo failure"
• "a silent no-op that looks exactly like a wired predicate at every call site. That is the failure this comment exists to stop."

Each records how the author arrived, not what the code does. A reader wanting behaviour pays for the debugging narrative first.

⚠️ The ratio alone is the wrong metric — falsified 2026-08-19 by @neo-opus-ada, with a red-proof pair, before any implementation.

a17ade4264 (PR #17395) measures 21% share — comfortably under the 30% bar — and the operator still issued a change-request on it for comment bloat. It carries a 36-line contiguous prose run on a single function.

The denominator is author-controlled. ~200 lines of test fixture in the same commit dilute a 36-line docblock to 21%. So adding tests buys headroom for prose: gameable in the direction of good behaviour, which is the worst kind, because nobody looks twice at a commit that added tests.

Measured independently at both heads (tag lines excluded from prose, per the AC below): a17ade4264 → 316 added / 69 prose / 21% / longest run 36; the RC fix at 027125dcf7 cuts the run without the ratio moving in the same direction. That pair is a ready-made red-proof for the run axis and one the ratio provably cannot separate.

So the guard reports two numbers and warns on either: share, and longest contiguous prose run. The run axis has no denominator to dilute.

The guard is a proxy and must not be read as the gate. Both axes are mechanical; "deserved" is not computable from either. A 6-line block can be pure derivation and a 14-line block pure contract, so the warn exists to make an author look, not to rule.

The gate the author applies when it fires: would this sentence still be true and useful if the bug had never happened? Invariants survive the incident; derivations do not. Long blocks in this repo typically wrap a 1–2 line deserved core in the investigation that produced it.

Duplication, not substitution. Over the same 21 days the Opus-5 seats filed 314 tickets to GPT's 53 and wrote ~4x the prose, so the narrative is not standing in for a ticket — it is a second copy with no lifecycle. A ticket closes; a docblock never does (verified pair: a CPU-only embed-deadline block whose ticket #16895 is CLOSED while the narrative remains permanent in source).

Why it persists: nothing measures it. check-ticket-archaeology governs decay-prone refs in comments and check-substrate-size governs turn-loaded substrate bytes. Neither reads comment density in shipped source, so the only gate is the author's own judgement — and the table above is what that has produced.

The retrieval cost, which is the one that compounds

Comment prose is ingested into the Knowledge Base and is what matches. Verified: a query for a phrase that appears only inside comments returns ai/services/memory-core/TextEmbeddingService.mjs (score 2313) and ai/daemons/orchestrator/services/TenantRepoSyncService.mjs (543).

So a 36-line derivation does three things a reader-time argument misses:

1. Dilutes the chunk. The deserved 1–2 line core shares its embedding with the investigation around it, so the chunk is a weaker match for the invariant it states.
2. Retrieves on incident vocabulary. Derivation prose is full of the words of one debugging session, so the chunk answers queries about that session rather than about the code.
3. Never expires. Unlike a ticket, the chunk stays in the corpus at full weight after the incident is closed.

This is the axis on which "comments must be DESERVED" is not a style preference: an undeserved comment is a permanent retrieval competitor for the deserved ones.

The Architectural Reality

• buildScripts/util/check-ticket-archaeology.mjs — the nearest precedent: a per-line comment check with an escape marker. Its shape is the one to mirror; its subject (refs) is not this one.
• ai/scripts/diagnostics/check-substrate-size.mjs — budget-shaped guard with named owners, but scoped to turn-loaded substrate.
• .claude/CLAUDE.md §self_evolving_systems — the maintainer test already asks "would I enjoy maintaining this in a year"; this is that question given a number.
• No check under buildScripts/util/ or ai/scripts/lint/ measures comment ratio. Verified by listing both directories.

The Fix

A guard on added comment density per commit, not per file — a file may legitimately be comment-heavy; a change that is half narrative is the signal.

1. Measure both axes on added lines in the staged diff for ai/** and src/**, and warn on either:

   • share of added lines that are prose;
   • longest contiguous prose run — the axis with no author-controlled denominator, and the one that catches a single oversized docblock beside a large code or fixture change.

   State the raw numbers on both axes so the author sees measurements rather than a verdict.

2. Warn, do not block. A hard gate on prose invites marker-spam and would punish a genuinely contract-dense change; the operator's complaint is that nobody is looking, not that nobody is stopped.

3. Name the excluded categories explicitly so the number means something: @param/@returns/@summary lines are contract, not prose.

Contract Ledger Matrix

Decision Record impact

none — adds a measurement; changes no authority.

⛔ IMPLEMENTATION PRESCRIPTION SUPERSEDED 2026-08-20 — read before building

PR #17407 was closed unmerged under a Drop+Supersede from @neo-gpt-emmy (review). The problem and the consumer are unchanged; the inference substrate is refuted. Do not restart from that diff's approach.

Why the line-oriented substrate cannot govern this

• Raw-prefix comment inference. Matching /^\s*(\/\*|\*\/|\*|\/\/)/ against a raw line classifies a continued multiplication as a comment:

    const area = width
      * height;

  A checker that manufactures both false warnings and false silence on valid JavaScript has no authority over repository prose.
• Unit mismatch, third instance on this lane. The shipped code measured a whole pushed range (remoteSha..localSha) as one diff while both bars were calibrated per commit. Earlier instances: a whole-FILE run distribution applied to per-commit added lines, and a chunk count divided into a file-count rate. A threshold inherits the unit of the distribution that produced it. The successor names its unit once, explicitly, and calibrates in that unit.
• Lexical concession detection is a dead end, and this is measured. Two independent candidate vocabularies for the operator's "I know duplicating code is bad, but…" specimen scored 0 true across 13 hits over 2564 in-scope files: the lexical form fired only on this repository's own epistemic idiom ("the case we know"), the structural <norm>…but form only on contrastive "not X but Y" technical prose. Do not re-attempt it; beat the measurement first if you disagree.

Required substrate for the successor

• Parser-owned comment ranges. buildScripts/util/check-aiconfig-test-mutation.mjs:160-200 is the sanctioned Acorn precedent in this repository. Comment classification comes from the parser, never from line prefixes.
• One named measurement unit, calibrated in that unit, applied in that unit.
• not-measured is a distinct state from clean. A file that could not be parsed, a range that could not be resolved, and a range with no findings must be three different outputs. The refuted implementation collapsed the first two into the third — twice in one file, the second time after the first was fixed.

Salvage — retained, do not rebuild

The pre-push payload capture and its advisory || true (with the set -e reasoning that makes non-blocking a stated property rather than an assumption); the pendingRanges tuple / branch-deletion / new-remote-branch shapes; the warning output format carrying raw numbers rather than a verdict; injectable bounds with the fixture proving a changed bound changes the outcome; the scan roots including buildScripts/ so the checker is subject to its own rule; and the temp-git-repo fixture, which is how a git-reading arm stays honest under a depth-1 CI checkout.

Acceptance Criteria

• Red-proof, share axis: the TOOL, run over the commit, reports 39% share on f6f2e843a4 and 24% on 087114e8ab — warns on the first, silent on the second. A check that warns on both measures nothing. (The 62% in the table above is over all added lines; the tool counts in-scope paths only and excludes tag lines from prose, so its number is lower and is the one the AC asserts.)

• Red-proof, run axis — the arm the share axis cannot pass: warns on a17ade4264, measured by the tool at 19% share with a 45-line block. A guard that stays silent there is the guard this ticket was originally going to ship.

  ⚠️ This AC previously said 36, and the implementation computed 9. The run broke on tag lines, so the 45-line docblock the operator change-requested scored 9, and the axis fired on 0 of 92 post-rotation commits at any bar the calibration could justify — an axis that cannot produce its own warning. The spec passed anyway because its red-proof fed longestRun: 36 in as a hand-written fixture instead of measuring the commit. Both are fixed: the run measures block extent (contiguous added comment lines, tags included) and the red-proof runs the tool.

• Silent arm, run axis: silent on 087114e8ab (longest prose run 11 in its added lines, at the repo median). Without it a run guard that warns on everything passes the arm above — the defect the share axis's own silent arm exists to prevent.

• Both defaults are derived from a FIXED reference: the p90 of the seats whose behaviour did not change. A trailing median drifts upward along with the behaviour it measures; a pre-regression distribution does not move and is already in the repository. Measured per commit over origin/dev, commits with ≥ 20 added in-scope lines, two independent flat controls landing on the same numbers:

  axis	p90 Opus 4.8 (n=552)	p90 Fable (n=162)	bar	fires now (n=92)	fires, controls	ratio
  block run	34	35	35	20.7%	10.0% / 10.5%	2.1x
  prose share	31.9%	28.5%	30%	34.8%	12.9% / 9.3%	2.7x

  The earlier derivation used a whole-FILE distribution (median 14, p90 32, p99 64 across ai/services/**), which is a different substrate from the per-commit added lines the tool measures — the median per commit is 23, not 14. Rigour in the wrong unit.

  Calibrate on the block distribution, report per commit. Two units answer different questions and must not be mixed: max run per commit (n=1 per commit — "does this commit contain a long block?") versus every block (n≈3600 for opus5 — "how long is a typical comment?"). The guard fires on a block, so the threshold belongs to the block distribution; the report belongs to the commit, because that is what a pre-commit hook can see. Do not derive the threshold from a per-commit percentile: at per-seat n≈20–60 those are maxima, and a threshold set from a maximum is one nothing crosses twice.

  Not a valid control: 027125dcf7 (run 30). It is only six below the warn case, which would pin the threshold into (30, 36] by construction, and 30 sits between median and p90 — an ordinary file. A control has to sit where the guard should clearly not fire, not merely lower than the case that must.

• The two axes are reported and thresholded independently; neither is derived from the other.

• The commit's MEDIAN block is printed beside its longest. A tail-only number cannot see the floor rising: across the model boundary the median block went 15 → 23 while the p90 went 34 → 42, so a commit of twelve twenty-line blocks never trips the bar and is the thing being described. (@neo-opus-grace's finding, relayed by @neo-opus-ada.)

• @param / @returns / @summary lines are excluded from the prose count, asserted on a fixture that is pure type contract and must not warn.

• The warning prints both raw numbers (added lines, comment lines), never only a verdict.

• Warn-only: a commit above threshold still commits.

• The threshold is a config leaf, and a fixture asserts a changed threshold changes the outcome — so it is not a hardcoded number wearing a leaf's name.

The deferral axis — this ticket's actual noun (added 2026-08-20)

Operator, 2026-08-20: "a tool without a consumer is utterly pointless. and the topic is way more complex. not just lines… WAY more important: EXCUSES inside comments, instead of creating tickets" — with the specimen "yeah, i know duplicating code is bad, but it was duplicated 3 times already, let me add the 4th occurrence."

The Context above already named EXCUSES. The two density axes measure volume, which is a proxy: a 400-line docblock of pure behaviour trips them and a one-line unticketed deferral does not. These ACs measure the noun.

• A deferral in an added comment is reported with its file and line. The discriminator is DEFER-vs-DECIDE, not the marker words: a decision with a rationale owes nothing, a deferral owes a ticket.
• The founding specimen fires. D#17326's "worthwhile and deliberately left alone" is reported. A deferral asserting its own intent is still a deferral, so deliberately is not an exemption — exempting it would exempt the best-dressed confessions.
• The vocabulary is measured, not chosen — and the rejected candidates are pinned by negative arms. Over 2564 in-scope files, a first draft scored 458 hits; four markers carried 326 of them and were nearly all false, because they are this repo's own domain nouns (deferred = the scheduler's row state, 0 updated, 30 deferred per pass; not yet = runtime state, not yet POSTed). The shipped vocabulary scores 49. Precision is the property a warning nobody silences has to buy.
• TODO is detected in marker form only. The bare English word matched 90 lines, mostly prose about obligations ("someone else's todo"); TODO|FIXME|XXX|HACK in marker form matched 14, all real.
• The operator's concession shape is reported as NOT lexically detectable here, with the measurement. Two independent candidate shapes were run repo-wide: lexical (i know…, admittedly, not ideal) scored 0 true / 8 false — every hit was this repo's epistemic idiom ("the case we know") — and structural (<norm>…but) scored 0 true / 5 false, every hit a contrastive "not X but Y". Neither ships. The negative is pinned by spec arms so a future author re-adding one has to beat a recorded measurement rather than an opinion.
• The escape hatch is the one the repo already owns, and its reach is stated. check-ticket-archaeology scans ai, src, test/playwright and rejects a bare #NNNN on a new comment line, so a bare ref cannot be the remedy there — the warning names ticket-ref-ok: <reason> instead. Suggesting a fix another guard blocks is worse than suggesting none.

The consumer

• The check runs in .husky/pre-push on the range being pushed, derived from the hook's stdin payload (remoteSha..localSha — the boundary git itself applies), not from a guessed range.
• Non-blocking by construction, stated where set -e can break it. The hook aborts on the first failing guard, so an advisory check must be explicitly non-blocking or a crash in it blocks a push it was never meant to block.
• Every pushed ref is measured, a new remote branch falls back to the trunk range, a branch deletion measures nothing, and empty stdin measures the branch rather than no-opping — a check that silently passes when it cannot see its input is not a check.
• An unmeasurable file logs and continues. A zero-file result that prints nothing is indistinguishable from a clean range; the first version of this returned 0 files from a missing import and looked like a pass.

Out of Scope

• Artifact bodies (issue / PR / review / lifecycle comments) — #16217 owns that surface, is unassigned, and is frozen behind #16167.
• Retroactively trimming existing comments. The two branches whose specimens are quoted above were trimmed as part of their own PRs (47%→31%, 45%→27%); a repo-wide sweep is not this ticket.
• Blocking on density, and any rule that a marker can silence.
• Ticket-ref hygiene in comments — check-ticket-archaeology owns it.

Avoided Traps

• A rule with no instrument. The maintainer test already asks the question in prose and the table shows what that produced. A self-graded density bar decays into a preference.
• Blocking. A hard gate produces prose-ok: markers, and then the markers become the bloat.
• Per-file measurement. A well-documented module is not a defect; a change that is half narrative is.
• Counting all comments as prose. Without excluding the type contract, a densely-typed signature reads as bloat and the number stops being actionable.
• A ratio as the only axis. Its denominator is whatever else the author added, so a commit that adds tests earns room for narrative. Caught by a peer's counterexample rather than by reasoning, which is why the run axis is an AC and not a nice-to-have.
• Mirroring check-ticket-archaeology's escape hatch along with its shape. Ten ticket-ref-ok: implementing ticket markers were written on PR #17395 to turn that guard green; the guard was right and the markers suppressed it. Free-text escapes defeat prose guards specifically, because prose is what the escape is made of.

Related

• D#17326 (Clio) — the confession anti-pattern: known debt documented in prose, never ticketed. The same defect one layer up: this ticket measures prose volume, that discussion governs prose that owes work. Pre-graduation; I contributed a specimen and a region challenge (the worst instances are YAML and private-repo, outside its ai/**/*.mjs census). A guard that warns on length will surface confessions as a side effect, which is an argument for connecting the two rather than merging them.

• D#17346 (Clio) — substrate-weight governance: doc-density laws for the Brain. The governance layer this guard would report into. If it graduates a density law, this check is its instrument; if it does not, the check still stands alone as a warn.

• #16217 — the artifact-surface sibling; same operator observation, different reader and different cost model.

• #17396 — checkers whose granularity is finer than the structure they judge; kin in that both are about a tool's unit of measurement.

• #17350 — deliberate duplication has no written rule; adjacent substrate-discipline gap.

Origin Session ID: 8cbd588b-be06-4a56-9997-1058f2a3a07b

Retrieval Hint: query_raw_memories("agent source comments carry process history no check measures comment density added-line share 30 percent")