LearnNewsExamplesServices
Frontmatter
id17679
titleBrowser launch receipts are overwritten before native correlation
stateClosed
labels
bugdeveloper-experienceaitestingregression
assigneesneo-gpt-emmy
createdAtAug 24, 2026, 8:18 AM
updatedAtAug 24, 2026, 9:50 AM
githubUrlhttps://github.com/neomjs/neo/issues/17679
authorneo-gpt-emmy
commentsCount0
parentIssue16151
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 24, 2026, 9:50 AM

Browser launch receipts are overwritten before native correlation

Closed Backlog/active-chunk-19 bugdeveloper-experienceaitestingregression
neo-gpt-emmy
neo-gpt-emmy commented on Aug 24, 2026, 8:18 AM

Context

PR #16162 introduced a privacy-bounded browserLifecycle.launchExits[] receipt for rejected E2E browser launches. PR #17609 later enriched that row with channel, browser kind, launch mode, exact-token cause, and a bounded remedy.

The parent investigation #16151 now has new native evidence: a bounded Aug 23 sweep found 26/26 Google Chrome diagnostics in the _RegisterApplication → TransformProcessType → ChromeMain family, all Node-parented, all in the Codex/ChatGPT coalition, all SIGABRT, and all captured 21–57 ms after launch (median 39 ms). Those native reports survive; the reporter half needed to bind one report to project/profile does not.

Current E2E config writes one fixed reporter file:

test-results/e2e/benchmark-system-info.json

A read-only search across the shared agent, deployment, and Codex worktree roots found no surviving file. This does not identify whether cleanup, later-run overwrite, or another lifecycle removes it. The verified failure is narrower: after the run stream advances, the reporter join artifact is unavailable while native diagnostics remain.

The Problem

One fixed mutable receipt cannot retain a sequence of launch failures. Even when the reporter classifies correctly, a later run can replace the only file, and Playwright's run artifacts have their own cleanup lifecycle. The parent then owns a native PID/time receipt with no durable project/profile/browser-kind counterpart and cannot execute its first admissible correlation step.

This is a successor to resolved #16161 / PR #16162. Reopening that ticket would violate the resolved-ticket lifecycle rule; the retention contract is new evidence from the active run stream.

The Architectural Reality

  • test/playwright/playwright.config.e2e.mjs already owns run-scoped artifact isolation through NEO_E2E_RUN_ID, RUN_ID, and ARTIFACT_ROOT. Battery runs therefore have a source-owned run coordinate today.
  • test/playwright/e2e/custom-reporter.js owns receipt creation, immediate flush on launch exit, deduplication, and the privacy-bounded lifecycle row.
  • The reporter's configured output is a fixed sibling of the Playwright outputDir; it is not run-scoped and carries no retention policy.
  • test/playwright/unit/e2e/browserLifecycleReporter.spec.mjs exercises the real reporter with temporary files, but only one run/file at a time; it cannot detect cross-run overwrite.
  • Native macOS diagnostics are evidence inputs, not a storage target. Neo must not delete, suppress, or parse personal browser content from them.

The mandatory full structure map currently fails with Cannot create a string longer than 0x1fffffe8 characters. A scoped --root test/playwright/e2e --files --loc run succeeds and identifies custom-reporter.js (253 LOC) as the sole reporter sibling beside gl.setup.mjs and globalSetup.mjs; no new module or directory placement is prescribed.

The Fix

  1. Replace the single mutable reporter file with one privacy-bounded file per E2E run.
  2. Reuse NEO_E2E_RUN_ID when the caller supplies it. When absent, mint one opaque reporter run id at coordinator start and echo it in the receipt; never derive identity from a seat, URL, title, profile path, or launch command.
  3. Configure a lifecycle-receipt root outside the Playwright-cleaned outputDir while keeping it under the existing E2E test-results authority. The config owns the root; the reporter owns only per-run files beneath it.
  4. Add bounded reporter-owned retention over only its recognized receipt files. The implementation must state and test the chosen horizon/count rationale; unrelated artifacts are never pruned.
  5. Preserve the existing launch-exit schema and privacy negatives. The new run coordinate and capture time enable a local PID/time join; this ticket does not add a native-diagnostic parser.

Contract Ledger

Target Surface Source of Authority Proposed Behavior Fallback / Edge Case Docs Evidence
E2E lifecycle-receipt root playwright.config.e2e.mjs reporter configuration + existing run-scoped artifact pattern Explicit root outside the Playwright outputDir; one child file per run Invalid/unwritable root fails loud before product assertions; no fallback to the fixed file Config comment + reporter JSDoc Config import/control + two-run filesystem witness
Reporter run identity Caller NEO_E2E_RUN_ID or reporter coordinator Preserve a supplied run id; otherwise mint one opaque run id once and echo it Re-imports/workers do not fork one logical run into several identities Reporter JSDoc Supplied-id and minted-id controls
browserLifecycle.launchExits[] retention custom-reporter.js Existing privacy-bounded row persists in its run file immediately on capture Later runs never overwrite prior files; write failure is loud and cannot erase the test failure Existing row docs + retention note Two sequential reporter runs; first incident survives second
Receipt pruning Reporter-owned filename/schema marker Bounded retention removes only expired/excess recognized receipt files Malformed, symlinked, foreign, or unrelated files are untouched and reported/ignored safely Pruner JSDoc Positive prune + foreign-file negative control
Native correlation consumer #16151 evidence workflow Run id, receipt timestamp, project/profile/browser kind, process id, exit code/signal, cause/remedy, and transport state remain available long enough for local correlation No public PID/path/browser-content projection; absence remains unknown, never evidence of no crash #16151 receipt guidance Synthetic PID/time join over two retained runs

Decision Record impact

none. This preserves Playwright as the sole browser owner and extends the existing local test-artifact contract.

Acceptance Criteria

  • A pre-fix control proves two reporter runs aimed at the fixed output path cannot preserve both launch-exit receipts.
  • Two post-fix sequential runs create distinct receipt files, and the first launch exit is byte-readable after the second completes.
  • A supplied NEO_E2E_RUN_ID is preserved exactly; an unset value produces one opaque run id shared by the receipt for that logical run.
  • The lifecycle-receipt root is configured outside the Playwright outputDir; cleaning/recreating the run artifact directory does not remove retained lifecycle receipts.
  • Retention is bounded with a documented rationale and prunes only reporter-owned receipt files; foreign files and symlinks survive unchanged.
  • Immediate flush on rejected launch remains load-bearing: a run that aborts before its first assertion still leaves its per-run receipt.
  • The serialized row remains negative for executable path, launch args, user-data directory, URL, title, profile content, and raw error text.
  • Existing channel/browser-kind/launch-mode/cause/remedy and transportState: not-observable contracts remain green.
  • The retained fields are sufficient for a local synthetic PID/time correlation; no native diagnostic parser or public process identifier is added.
  • Existing E2E config/reporter unit coverage and focused reporter tests remain green.

Out of Scope

  • Diagnosing or repairing Chrome's AppKit registration abort.
  • Running the concurrent same-bundle lifecycle cell from #16151.
  • Parsing, deleting, suppressing, or relocating macOS diagnostic reports.
  • Adding a browser preflight or any second browser owner.
  • Persisting raw launch commands, browser content, profile paths, or native diagnostic payloads.
  • Adding E2E to CI.

Avoided Traps

  • Reopening #16161. Its resolved normalization contract remains valid; this is new cross-run retention evidence.
  • One “latest” file with richer fields. More fields do not fix overwrite; correlation requires temporal multiplicity.
  • Storing under the cleaned run artifact directory. That recreates the loss under a new filename.
  • Unbounded incident accumulation. Privacy-bounded rows still require an owned retirement rule.
  • Guessing native ownership from the Codex coalition. Coalition scope does not identify project, profile, sandbox authority, or logical run.

Related

Parent: #16151. Baseline: #16161 / PR #16162. Field enrichment: #17595 / PR #17609. Evidence-routing sibling: #17605 / PR #17606.

Live latest-open sweep: checked the latest 20 open issues immediately before filing; no equivalent receipt-retention ticket found. Exact and semantic searches found only #16151, resolved #16161, and field-enrichment #17595.

A2A in-flight sweep: checked the latest 100 messages across read states immediately before filing; no competing retention claim found.

Origin Session ID: 0dc1379e-5329-4fba-80ca-f6466822f7c9

Retrieval Hint: browser lifecycle launch-exit receipt per-run retention native PID time correlation benchmark-system-info overwrite

tobiu referenced in commit 4b529ed - "fix(testing): retain browser launch receipts per run (#17679) (#17680)" on Aug 24, 2026, 9:50 AM
tobiu closed this issue on Aug 24, 2026, 9:50 AM