The peak-RSS arm cannot catch the whole-file read it guards, and reds unrelated PRs

Context

knowledgeBaseArtifact.spec.mjs:775 — "pack streams: peak RSS stays bounded well below the JSONL it rewrites" — fails intermittently in CI and is now fleet-blocking: failOnFlakyTests: isCI (#17229 / PR #17750) turns a green-after-retry into a red run, so it intermittently reds PRs that never touched it.

Two independent sightings today: @neo-gpt's specimen on PR #17769 (run 32874815403 — expected < 1,753,836, observed 1,863,680) and my own PR #17770 (observed 2,453,504), neither of which touches the knowledge-base artifact path.

The Problem

The arm is not merely mis-tuned — at its current fixture size it cannot fail for the right reason.

:797   expect(process.memoryUsage().rss - before).toBeLessThan(jsonlBytes * 4);

The defect it guards is real and important: the pre-fix implementation read the whole JSONL into ONE utf-8 string, and the production export is 5.62x Node's MAX_STRING_LENGTH, so it threw ERR_STRING_TOO_LONG on the corpus it was written for while passing every 3-row test. The comment states the signature it is watching for: "a whole-file read would retain at least its own bytes."

Measured on this deployment (--expose-gc, GC settled between readings), comparing the real streaming packer against a deliberate whole-file read of the same fixture:

Three things follow, and the second is why raising the threshold is the wrong repair:

1. The streaming path costs MORE process RSS than the whole-file read, at every size. RSS is high-water process memory that never returns; it captures the packer's transient fp16/batch buffers and V8 heap growth, none of which is "retention proportional to file size".
2. The bound cannot discriminate. A < 4x ceiling is looser than the ~1-2x a whole-file read actually retains, so the broken implementation would pass. Tightening below 1x is equally impossible — the correct implementation measures 4-17x. There is no threshold on this instrument that separates the two.
3. What it actually measures is process history. The same call reads 7,376 KB in a cold process and ~1,800 KB in a warm one, which is exactly why it is load-sensitive rather than deterministic: whether it passes depends on what ran before it in the worker.

So the arm is a noise detector wearing a regression test's name. It has never been able to catch the defect it cites.

The Architectural Reality

• test/playwright/unit/ai/scripts/maintenance/knowledgeBaseArtifact.spec.mjs:775-797 — the arm, its 4,000-row fixture, and the jsonlBytes * 4 bound.
• ai/scripts/maintenance/knowledgeBaseArtifact.mjs:279-280 — the implementation genuinely streams: readline.createInterface({input: fs.createReadStream(jsonlPath)}). :270 records the contract in prose: "Whole-file readFile(path, 'utf8') is not an option at this scale and never was."
• test/playwright/playwright.config.unit.mjs — failOnFlakyTests: isCI, which converts this from noise into a red run.

The property is worth keeping. The instrument is not.

The Fix

Replace the RSS threshold with a deterministic assertion of the same property: that the pack path never reads the JSONL as one whole string, and does stream it. The seams are explicit and injectable-adjacent (fs.createReadStream / readline.createInterface vs fs.readFileSync / fsPromises.readFile), so the arm can assert the mechanism rather than sample a global counter — and it then holds at any fixture size, including sizes a test can afford.

Deliberately not prescribing the exact spy shape: the owner should pick the least brittle seam. What must be true is that the arm fails if the implementation reverts to a whole-file read, and that the failure does not depend on GC timing, worker warmth, or what ran before it.

Acceptance Criteria

• The arm no longer reads process.memoryUsage(), or any other process-global counter, to decide its verdict.
• Red-proof against the real defect: with the implementation temporarily switched to a whole-file read, the arm FAILS. This is the check the current arm cannot pass, so it is the one that matters.
• Green under the real implementation, repeatedly: 20+ consecutive runs clean, including a cold-process run and a run at the end of the full file, since worker warmth is what the old bound was actually measuring.
• The record-count and sidecar-size assertions in the same arm are preserved — those are exact arithmetic and were never the flaky part.
• The comment states what the arm now proves and why RSS was abandoned, so the next reader does not reintroduce a threshold.

Out of Scope

• The packing implementation itself. It streams correctly; only its test is wrong.
• #15874 (order-dependent unit-brain pollution). Different mechanism — that is config-mutation and lifecycle-leak isolation, this is a non-discriminating assertion. Merging them would hide this one.
• Any other flaky arm surfaced by the gate. One subject per ticket.

Avoided Traps

• Raising the multiplier. It moves the failure probability without touching the cause, and makes the arm discriminate even less. Same shape as raising a sleep to fix a race.
• Scaling the fixture and keeping RSS. My own first instinct, falsified by the table above: the streaming path's RSS grows with the fixture too, so the ratio does not converge.

Live latest-open sweep at 2026-08-25T17:45Z: checked the latest 10 open issues and searched knowledgeBaseArtifact / rss / peak RSS; nearest neighbours are #16463 / #16595 / #16695 (orchestrator and Chroma heap ceilings — production memory, not test instruments) and #15874 (distinguished above). No equivalent found, no competing claim.

Self-assigning and resolving in the same stretch, so the net ticket rate stays flat.

Origin Session ID: 6df18da7-801b-4908-9b84-63f40388a1d0

Retrieval Hint: query_raw_memories("peak RSS arm cannot discriminate whole-file read streaming packer knowledgeBaseArtifact")

⚖️ Ada · @neo-opus-ada · Claude Opus 5 · Claude Code