E2E batteries clobber failing runs' traces and drop webServer stdout

Context

Surfaced 2026-08-08 during the #16620 leg-2 battery census (5 consecutive post-guard WorkstationFiveBeatNL runs, headed Chromium, --workers=1, head f59c249a03). Run 1 went red 1.3s into boot (net::ERR_ABORTED on the initial page.goto — a boot/navigation seam, classification undetermined between browser-process transient and dev-server socket race). Runs 2–5 passed. The census comment on #16620 records the full receipt.

The battery produced a red and then destroyed the evidence that would have classified it: the failing run's trace.zip was overwritten by a later passing run, and the dev server's stdout was never captured. AC-3 of #16620 names this follow-up.

The Problem

Two evidence classes die exactly when a roving red appears:

1. Trace clobbering. The e2e config pins one fixed artifact root (test/playwright/playwright.config.e2e.mjs:33 — outputDir: './test-results/e2e/artifacts') and trace: 'on' (:48) writes every execution's trace into the same per-test-name directory. Serial run N+1's execution of test T overwrites run N's trace.zip for test T — passing runs included, since trace: 'on' writes unconditionally. A battery therefore keeps only the LAST run's trace per test, which is usually the passing one.
2. Server-log loss. The webServer block (:51-57) has no stdout/stderr persistence, so the dev server's own output is absent from every run's artifacts. Boot-seam failures — where a server-side socket race is a live candidate — leave zero server-side evidence.

Solo runs cannot reproduce the load-class failures batteries exist to catch (#16620's suite-load faces; this boot transient). The harness must not clobber the evidence those batteries exist to keep.

The Architectural Reality

• Per-worker isolation precedent (unit-test world): test/playwright/configTemplateResolver.mjs derives worker-scoped directories from TEST_WORKER_INDEX — isolation by default. The e2e battery world has no run-scoped equivalent: --workers=1 serial executions share both worker index and outputDir.
• Append-only precedent: test-results/dock-wow-demo/PROJECT_RECORD.md — film production records are never overwritten.
• Playwright levers are sufficient: --output <dir> overrides outputDir per invocation; the webServer block supports stdout/stderr piping. The fix is config/invocation-level — no new subsystem, and per Playwright's ownership of the per-test layout, run-scoping belongs at the outputDir level, not inside test-dir naming.

The Fix

1. Run-scoped artifact isolation: make the e2e outputDir env-overridable (e.g. NEO_E2E_RUN_ID → test-results/e2e/battery-<id>/run-N/artifacts), so every execution's trace lands in a directory no later run can touch. Config-level default so isolation is the shipped posture, not a remembered flag; --output remains the manual override.
2. webServer stdout/stderr capture: persist the dev server's stdout/stderr into the same per-run directory (webServer.stdout/stderr: 'pipe' plus capture into the run dir, or the battery wrapper teeing the server process output), so a boot-seam red carries its server log.
3. Prefer config + documented invocation (file-free). If a battery-runner wrapper script is introduced instead, structural-pre-flight fires for the new file at implementation time.

Contract Ledger

Acceptance Criteria

• Two consecutive runs of the same spec — first staged-failing, second passing — leave BOTH runs' trace artifacts intact under run-scoped directories.
• A run's webServer stdout/stderr is captured into that run's artifact directory.
• Isolation is default-on for battery invocation (env/config-level); solo and film-take invocation behavior unchanged (film mode self-excludes instrument fixtures per PR #16726).
• The per-run convention is named where the next battery inherits it (config comment or the battery discipline note).

Out of Scope

• The boot-seam defect itself (undetermined: browser-process transient vs dev-server socket race) — this ticket makes its next occurrence attributable; it does not diagnose it.
• CI wiring for e2e (there is no CI e2e job; host-battery discipline per #16620).
• Per-worker unit-test isolation (already shipped via TEST_WORKER_INDEX).

Avoided Traps

• Do NOT substitute trace: 'retain-on-failure' for run isolation: retention policy still writes into the shared per-test directory, and a later failing run of the SAME test still overwrites the earlier one. Retention ≠ run isolation — and it drops the always-on traces batteries use for continuity/film analysis.
• Do NOT timestamp inside Playwright's per-test directory naming: Playwright owns that layout; run-scoping belongs at the outputDir/invocation level.

Related

#16620 (origin — leg-2 census evidence gap; AC-3 follow-up), PR #16701 (merged dwell-reverify guard), #16499 (receipt-unmask precedent), PR #16726 (film-take self-exclusion — sibling harness-integrity shape).

Sweeps

• Live latest-open sweep: checked latest 20 open issues at 2026-08-08 ~18:52Z — no equivalent (closest: #16620 the parent; #16684 lint-workflow watch parity — different domain).
• A2A in-flight sweep (30-message window, ~90min herd window): active claims on #16684 / #16695 / #16690 / #16715 / #16723 / #16507 / #16726 / #16731 — no overlap.
• KB semantic sweep (type=ticket): no equivalent; surfaced the TEST_WORKER_INDEX per-worker isolation precedent and the PROJECT_RECORD.md append-only precedent cited above.
• Local exact sweep (resources/content/issues, trace.zip|artifact overwrite|webServer stdout|per-run output): only closed #15048 (unrelated Fleet bootstrap).

Decision Record impact

none — test-harness config/invocation discipline; no ADR surface. Structure-map gate: N/A (test/playwright scope; not ai/ / Agent OS / MCP / skills / placement).

Origin Session ID: baf25fde-703a-4b04-aafd-e2f1c1070b5d

Retrieval Hint: query_raw_memories("battery trace artifact overwrite run isolation webServer stdout evidence capture")