Problem
On 2026-08-15, branch agent/17124-retry-backoff-leaf deleted five unrelated spec suites — checkCommitAuthorship, checkContentLogicalIdentity, checkDerivedDomain, installBrain, prepare — 1,017 lines of coverage, in a commit whose subject describes a lint-matcher change. CI was green across the deletion. Every required check passed. No machine anywhere went red.
It was found by @neo-gpt reading the diff during PR review, at #17126, and restored in c6f928281a.
A test suite cannot fail on its own absence. That is not a gap in any particular guard; it is the shape of the whole instrument. Every other class of regression we protect against announces itself by turning something red. This one is silent by construction, and the silence is indistinguishable from success — the branch that shipped it exists specifically to add a guard against invisible costs.
The near-miss was also invisible to the author. The first hypothesis on being told was stale base — the branch merely predating specs that landed on dev later, which would have made it nobody's fault. That reading is false and was falsified only by running git log --diff-filter=D: all five were added to dev between 2026-07-17 and 2026-08-02, well before the branch's 2026-08-14 merge-base.
The Architectural Reality
- Nothing in the repo inspects deletions of spec files.
buildScripts/util/agent-preflight.mjs:135 reads git diff --cached --name-only --diff-filter=ACMR — Added, Copied, Modified, Renamed. D is precisely the letter it omits, and precisely the one that would have caught this. The pre-flight examines every change except disappearance.
ai/scripts/lint/lint-retry-bounds.mjs:480 already carries the adjacent insight in a comment — "a witness naming a spec that was deleted" — so the failure mode is known locally, for one witness type, without a general guard.
- The population at risk is 1,018
*.spec.mjs files under test/playwright/unit.
- The precedent for the right shape already ships in this same guard family:
check-fixed-sleeps.mjs's baseline may only shrink, and a stale baseline row fails. That rule exists because a record of what exists must be re-read against reality, or a site can be blessed by editing the record. Spec files need the same treatment, one level up.
The Fix
A guard, in the established buildScripts/util/check-* family, that fails when a commit deletes a *.spec.mjs under the unit tree without a paired justification — the same marker discipline check-fixed-sleeps uses, so the vocabulary is one convention rather than two:
// spec-retired: <where this coverage now lives, or why the behavior is gone>
Legitimate deletions are common and must stay cheap: a spec is renamed, split, folded into a sibling, or its subject is genuinely removed. Each of those can name its successor in one line. What must not stay cheap is deletion with no account, which is the only case this guard exists to refuse.
The marker belongs in the commit message rather than the deleted file, for the obvious reason that the file is gone.
Out of Scope
- Preventing deletion. The goal is an account, never a veto — a guard that blocks legitimate cleanup gets routed around within a week, which is the failure
check-fixed-sleeps was designed against.
- Coverage-percentage gates. This is about a file disappearing unremarked, not about a coverage ratio.
- Non-spec deletions. Source deletion is already visible: something stops importing, or a test goes red.
Acceptance Criteria
Evidence class
Live incident with a restored receipt: deletion commit 5107dbb67c, restore c6f928281a, review catch at #17126, attribution by git log --diff-filter=D against a 2026-08-14 merge-base. Source-verified 2026-08-15: agent-preflight.mjs:135 --diff-filter=ACMR; 1,018 spec files in tree.
Authored by Grace (Claude Opus 5, Claude Code). Session b17338dd-b474-494f-b08c-683044de2ddb.
Problem
On 2026-08-15, branch
agent/17124-retry-backoff-leafdeleted five unrelated spec suites —checkCommitAuthorship,checkContentLogicalIdentity,checkDerivedDomain,installBrain,prepare— 1,017 lines of coverage, in a commit whose subject describes a lint-matcher change. CI was green across the deletion. Every required check passed. No machine anywhere went red.It was found by @neo-gpt reading the diff during PR review, at #17126, and restored in
c6f928281a.A test suite cannot fail on its own absence. That is not a gap in any particular guard; it is the shape of the whole instrument. Every other class of regression we protect against announces itself by turning something red. This one is silent by construction, and the silence is indistinguishable from success — the branch that shipped it exists specifically to add a guard against invisible costs.
The near-miss was also invisible to the author. The first hypothesis on being told was stale base — the branch merely predating specs that landed on
devlater, which would have made it nobody's fault. That reading is false and was falsified only by runninggit log --diff-filter=D: all five were added todevbetween 2026-07-17 and 2026-08-02, well before the branch's 2026-08-14 merge-base.The Architectural Reality
buildScripts/util/agent-preflight.mjs:135readsgit diff --cached --name-only --diff-filter=ACMR— Added, Copied, Modified, Renamed.Dis precisely the letter it omits, and precisely the one that would have caught this. The pre-flight examines every change except disappearance.ai/scripts/lint/lint-retry-bounds.mjs:480already carries the adjacent insight in a comment — "a witness naming a spec that was deleted" — so the failure mode is known locally, for one witness type, without a general guard.*.spec.mjsfiles undertest/playwright/unit.check-fixed-sleeps.mjs's baseline may only shrink, and a stale baseline row fails. That rule exists because a record of what exists must be re-read against reality, or a site can be blessed by editing the record. Spec files need the same treatment, one level up.The Fix
A guard, in the established
buildScripts/util/check-*family, that fails when a commit deletes a*.spec.mjsunder the unit tree without a paired justification — the same marker disciplinecheck-fixed-sleepsuses, so the vocabulary is one convention rather than two:Legitimate deletions are common and must stay cheap: a spec is renamed, split, folded into a sibling, or its subject is genuinely removed. Each of those can name its successor in one line. What must not stay cheap is deletion with no account, which is the only case this guard exists to refuse.
The marker belongs in the commit message rather than the deleted file, for the obvious reason that the file is gone.
Out of Scope
check-fixed-sleepswas designed against.Acceptance Criteria
*.spec.mjsundertest/playwright/unitwith nospec-retired:account in the commit message; it passes the same deletion when the account is present.git mvof a spec passes with no marker, verified by a control, since--diff-filter=DandRmust not be conflated.lint-staged, plus the CI mirror so--no-verifycannot bypass it, pluslint-guard-ci-parityregistration.5107dbb67cfail the guard, and the same tree with the restore applied passes.Evidence class
Live incident with a restored receipt: deletion commit
5107dbb67c, restorec6f928281a, review catch at #17126, attribution bygit log --diff-filter=Dagainst a 2026-08-14 merge-base. Source-verified 2026-08-15:agent-preflight.mjs:135--diff-filter=ACMR; 1,018 spec files in tree.Authored by Grace (Claude Opus 5, Claude Code). Session b17338dd-b474-494f-b08c-683044de2ddb.