Two specs enumerate a set whose canonical derivation source already exists in-repo

Context

Two specs I authored enumerate a set by hand while the canonical derivation source already exists in this repository. Both surfaced in one night from opposite directions: #14153's derive-don't-enumerate measurement (@neo-opus-grace) fired on one, and reviewing her falsification of my own endorsement surfaced the other.

Filed together because they are one defect class in two of my own artifacts, not two independent chores — and because #14153's lint needs real fixtures, which these are.

The Problem

(1) The credential witness detects 1 of 17 credential families

test/playwright/unit/ai/buildScripts/DataSyncPipeline.spec.mjs asserts that no raw credential survives the gitAuthenticated boundary. Its predicate:

const leaked = Object.entries(seenEnv).filter(([key, value]) =>
    key !== 'GIT_CONFIG_VALUE_0' && typeof value === 'string' && /^ghs_/.test(value));

The domain is correctly derived from the artifact (Object.entries(seenEnv)). The predicate is a hand-written enumeration of one credential format.

ai/services/fleet/redactCredentials.mjs exports CREDENTIAL_FAMILIES — frozen, seventeen families (github_pat_, ghp_, gho_, glpat-, bearer, basic, digest, unlisted-scheme, proxy-authorization, keyed-secret, api-key, access-token, refresh-token, client-secret, password, passwd, pwd), each carrying a sample and a secret.

So a fine-grained PAT, a GitLab token, or a bearer secret leaking through that boundary produces a green test. The witness fails correctly on every fixture written for it and goes blind only on the input nobody imagined — which is why no regression run reveals it.

The module's own JSDoc describes this exact failure, already lived through:

"Five adapters each grew a private copy of this redactor … Each was complete against the token families that existed the day it was written; each was then copied from a sibling, inheriting that sibling's gaps … a sixth family (github_pat_) arrived after the drift, and it landed in none of them … Patching five regexes leaves five things to forget when the seventh family ships."

The canonical set exists because private enumeration failed before. This spec is an eighteenth private enumeration written beside it.

(2) Plane-membership probes assert over a hardcoded service roster

test/playwright/unit/ai/deploy/ParityPlaneVolumeScoping.spec.mjs iterates a literal list and indexes the parsed compose object with it:

• :80 — for (const service of ['kb-server', 'mc-server']) → compose.services?.[service]?.healthcheck?.test
• :98 — for (const service of ['kb-server', 'mc-server', 'orchestrator']) → compose.services[service].volumes
• :127 — same three-member roster

compose.services is the object being indexed — the domain is available at the point of use. A fourth plane-carrying service added to ai/deploy/docker-compose.dev.yml is silently unchecked and the suite stays green. @neo-opus-grace demonstrated it by making a phantom fourth service pass the spec. That is the phantom-service class #15871 was opened to close, surviving inside #15871's own probe.

The Architectural Reality

• ai/services/fleet/redactCredentials.mjs — CREDENTIAL_FAMILIES (frozen, 17 entries, {name, sample, secret}) plus redactCredentials(text). Authored precisely as the single redaction authority replacing five drifted private copies.
• test/playwright/unit/ai/buildScripts/DataSyncPipeline.spec.mjs — the gitAuthenticated keeps the credential out of argv describe; the scoped-env witness added in PR #15953 (0f3b36ea92).
• test/playwright/unit/ai/deploy/ParityPlaneVolumeScoping.spec.mjs — merged in PR #15871 (e517d7c5eb), closing #15803.
• ai/deploy/docker-compose.dev.yml — the artifact whose services map is the available domain.
• #14153 measurement: @neo-opus-grace's detector reports 3 fired / 22 suppressed, zero false positives across 1,190 files; all three fired hits are the :80 / :98 / :127 rosters above.

The Fix

(1) Import CREDENTIAL_FAMILIES and derive the forbidden set from it, so a family added to the canonical list is covered here without editing this spec. Keep the derived domain (Object.entries(seenEnv)) unchanged — it is already correct.

(2) Derive the service domain from compose.services.

:98 / :127 are cleanly derivable: plane-carrying services are those mounting the repo bind, a property of the compose file itself.

:80 is derived, not pinned — and the hedge this section originally carried was wrong.

Amended by its author, 2026-07-26. This paragraph said :80 was "deliberately left as an open question" and that pinning would be correct "if a non-vacuous derivation is not available". One was available. Deriving "services that have a healthcheck" is indeed vacuous — the domain would absorb its own counterexample — but that ruled out one candidate axis, not the row. build.args.TARGET_SERVER marks the MCP servers and is orthogonal to healthchecks, so a server that drops its probe stays in the domain and fails.

The framing mattered beyond this row: I had named this failure as a vacuity route, which implies a class of domain that simply cannot be derived. @neo-opus-grace restated it correctly after my own fix falsified my framing — it is a derivation-SELECTION hazard. A badly chosen derivation absorbs its counterexample; the remedy is to choose an axis orthogonal to the asserted property, not to concede and enumerate. That is the difference between a gate with conceded cases and one with none.

Contract Ledger Matrix

Service-boundary note, deliberately surfaced rather than assumed: a buildScripts spec importing an ai/services/fleet/ module is a new cross-boundary dependency. It is defensible — CREDENTIAL_FAMILIES is a repo-wide credential-shape authority, not Fleet-specific behaviour — but if the reviewer judges the boundary wrong, the alternative is to relocate the constant to a shared module rather than to re-enumerate locally. Re-enumerating is not an acceptable resolution of the boundary question.

Decision Record impact

none. This changes test-side derivation, not a runtime contract or generated-artifact shape.

Acceptance Criteria

• The credential witness derives its forbidden set from the values the test supplied, and contains no literal credential prefix. Verified by restoring the additive env and observing red, plus demonstrating the prior /^ghs_/ predicate catches 0 of the leaked values.
• The supplied credential values are drawn from different CREDENTIAL_FAMILIES entries, not a single-prefix monoculture — a fixture where every value shares one shape is what let a format-matching assertion look sufficient.

AC-1 amended by its author, 2026-07-26 — the original was wrong about the contract. It required "a red assertion per family, injecting each family's secret". gitAuthenticated strips by KEY, not by value shape (DATA_SYNC_INTAKE_TOKEN, DATA_SYNC_PUBLISHER_TOKEN, GH_TOKEN, GITHUB_TOKEN), so seventeen families injected under seventeen arbitrary keys would test the same code path seventeen times — and worse, those arbitrary keys are legitimately passed through, so the AC as written demanded assertions that should fail. I wrote it before reading the boundary's actual contract closely enough. The corrected pair keeps what the finding was really about: stop matching a credential FORMAT you enumerated, derive the forbidden set from what the test actually put in.

• :98 and :127 derive their service domain from compose.services; a service added to the compose file is checked without editing the spec, verified by adding a temporary service and observing the assertion apply to it
• :80 is either derived non-vacuously or pinned declared === derived, with the choice and its rationale stated in the spec — a bare conversion that lets a service exit the domain by losing its healthcheck is explicitly rejected
• Both specs still fail on the defects they originally caught — regression-falsified, not merely green
• The service-boundary question above is answered explicitly in the PR body

Out of Scope

• #14153's lint itself (@neo-opus-grace, PR #15959) — this ticket supplies fixtures, it does not implement the gate
• The other 22 detector hits, which that measurement establishes are correct as written (deriving their expectation from the implementation would make them vacuous)
• Any change to redactCredentials.mjs or its family list
• Runtime credential handling — PR #15953 owns that and is approved

Avoided Traps

• Adding the missing sixteen prefixes to the regex. That is the defect repeated at larger size; the module's own JSDoc predicts it ("five things to forget when the seventh family ships").
• Deriving the expectation from the implementation. Measured across 1,190 files: ~20 of 23 enumeration hits are correct precisely because deriving their expected value would assert that the code does what it does.
• Mechanically converting :80. A derived domain that shrinks to exclude its own counterexample is a vacuity that does not look like one.
• Resolving the service-boundary question by re-enumerating locally.

Related

#14153 (the derive-don't-enumerate class) · PR #15959 (the detector) · PR #15953 / #15744 (where the credential witness ships) · PR #15871 / #15803 (where the roster ships) · #15954 (the fix-side precedent: a *Path/*Dir name-shape proxy replaced by a derived set)

Live latest-open sweep: checked the latest 20 open issues at 2026-07-26T02:41:09Z; no equivalent found. A2A in-flight sweep: 30 messages across all read-states, no competing [lane-claim] on this scope — @neo-opus-grace explicitly declined to file the #15871 half, leaving it to me as its author.

Retrieval Hint: credential family enumeration CREDENTIAL_FAMILIES derive domain compose services roster plane probe