Context
Live incident, 2026-07-16 ~07:55–08:05Z on the shared agent machine: four agents' npm run test-unit invocations wedged simultaneously at 0% CPU (checkouts: claude, opus-vega, fable, plus a worktree agent). Also retro-explains an earlier same-day exit-143 stall a reviewer hit and documented as a TOOLING_GAP during a PR review. Incident broadcast with the workaround went out at 08:06Z.
The Problem
test/playwright/playwright.config.unit.mjs isolates the test chroma's data dir per process (neo-chroma-unit-test-${process.pid}) but binds its port to a machine-global fixed default (NEO_CHROMA_PORT_TEST || 18180) with reuseExistingServer: false. The isolation is half-done:
- Two agents running unit suites concurrently contend for one port — the loser's webServer boot wedges.
- Worse, the failure is sticky: a runner that dies without its graceful shutdown orphans its chroma (observed: PID 23930, PPID 1, 29+ minutes old, dead runner's tmpdir, LISTENING on 18180). Because
reuseExistingServer: false forbids adoption, every subsequent unit run machine-wide wedges against the corpse — three runners sat ESTABLISHED against it, polling, at 0% CPU.
- Cross-checkout process cleanup is (correctly) permission-gated for agents, so an agent cannot self-heal the machine — only the operator can reap the orphan.
Verified workaround (unblocked a run immediately): NEO_CHROMA_PORT_TEST=18190 npm run test-unit -- <spec> --workers=1.
The Architectural Reality
- The config already models per-process isolation for the data dir — the port simply never got the same treatment.
- Playwright's
webServer needs the port number at config-definition time, so the derivation must happen synchronously in the config module (both command and url template from the same value — one-line blast radius).
- CI is unaffected (single runner,
workers: 1); this is purely the multi-agent shared-machine topology — which is the Agent OS's PRIMARY topology.
The Fix
In playwright.config.unit.mjs, derive the default port per process when the env override is absent — e.g. probe a free port synchronously (bind 127.0.0.1:0, read the assigned port, close, use) or derive 18180 + (process.pid % 512) with a bind-probe fallback on collision. Env override behavior unchanged (NEO_CHROMA_PORT_TEST still wins — CI and deliberate pinning keep working). Audit the sibling configs (integration, component, visual) for the same fixed-port pattern and apply the same derivation where a webServer exists.
Secondary consequence, worth stating in the fix's JSDoc: per-process ports make orphaned test chromas benign (an orphan squats only its own dead port + tmpdir) — the sticky machine-wide failure mode disappears even before any orphan-reaping hygiene exists.
Acceptance Criteria
Out of Scope
- Orphan-reaper hygiene for dead runners' chroma processes / tmpdirs (separate leaf if the benign-orphan consequence proves insufficient).
- e2e config port conventions (
NEO_E2E_PORT is caller-supplied by design).
Avoided Traps
reuseExistingServer: true — adopting a random live chroma crosses test isolation (shared collections between agents' runs; the per-PID dataDir exists precisely to prevent this).- A lockfile queue — serializes all agents' test runs machine-wide (slow) and introduces its own stale-lock class; isolation beats queuing.
Decision Record impact
none.
Related
Incident A2A broadcast 2026-07-16 08:06Z (workaround + operator orphan-reap ask) · surfaced during the PR #15211 review-response loop (#14500). Deliberately NOT milestoned: shared-machine dev-infra hygiene, not release-gate scope (D#15209 ledger discipline — explicit deferral note).
Live latest-open sweep: checked latest 12 open issues at 2026-07-16 ~08:07Z; no equivalent found. A2A in-flight claim sweep: herd-window claims (#15216 Vega, #15219 Vega, #15186 Euclid) all disjoint — clean.
Origin Session ID: 75ed6708-c66b-4989-862d-2286e87abbf1
Retrieval Hint: "unit test chroma fixed port 18180 orphan wedge per-process derivation"
Context
Live incident, 2026-07-16 ~07:55–08:05Z on the shared agent machine: four agents'
npm run test-unitinvocations wedged simultaneously at 0% CPU (checkouts: claude, opus-vega, fable, plus a worktree agent). Also retro-explains an earlier same-day exit-143 stall a reviewer hit and documented as a TOOLING_GAP during a PR review. Incident broadcast with the workaround went out at 08:06Z.The Problem
test/playwright/playwright.config.unit.mjsisolates the test chroma's data dir per process (neo-chroma-unit-test-${process.pid}) but binds its port to a machine-global fixed default (NEO_CHROMA_PORT_TEST || 18180) withreuseExistingServer: false. The isolation is half-done:reuseExistingServer: falseforbids adoption, every subsequent unit run machine-wide wedges against the corpse — three runners sat ESTABLISHED against it, polling, at 0% CPU.Verified workaround (unblocked a run immediately):
NEO_CHROMA_PORT_TEST=18190 npm run test-unit -- <spec> --workers=1.The Architectural Reality
webServerneeds the port number at config-definition time, so the derivation must happen synchronously in the config module (bothcommandandurltemplate from the same value — one-line blast radius).workers: 1); this is purely the multi-agent shared-machine topology — which is the Agent OS's PRIMARY topology.The Fix
In
playwright.config.unit.mjs, derive the default port per process when the env override is absent — e.g. probe a free port synchronously (bind127.0.0.1:0, read the assigned port, close, use) or derive18180 + (process.pid % 512)with a bind-probe fallback on collision. Env override behavior unchanged (NEO_CHROMA_PORT_TESTstill wins — CI and deliberate pinning keep working). Audit the sibling configs (integration,component,visual) for the same fixed-port pattern and apply the same derivation where a webServer exists.Secondary consequence, worth stating in the fix's JSDoc: per-process ports make orphaned test chromas benign (an orphan squats only its own dead port + tmpdir) — the sticky machine-wide failure mode disappears even before any orphan-reaping hygiene exists.
Acceptance Criteria
test-unitinvocations from different checkouts complete without port contention (no env vars set).NEO_CHROMA_PORT_TESToverride behavior unchanged (spec or documented manual check).Out of Scope
NEO_E2E_PORTis caller-supplied by design).Avoided Traps
reuseExistingServer: true— adopting a random live chroma crosses test isolation (shared collections between agents' runs; the per-PID dataDir exists precisely to prevent this).Decision Record impact
none.
Related
Incident A2A broadcast 2026-07-16 08:06Z (workaround + operator orphan-reap ask) · surfaced during the PR #15211 review-response loop (#14500). Deliberately NOT milestoned: shared-machine dev-infra hygiene, not release-gate scope (D#15209 ledger discipline — explicit deferral note).
Live latest-open sweep: checked latest 12 open issues at 2026-07-16 ~08:07Z; no equivalent found. A2A in-flight claim sweep: herd-window claims (#15216 Vega, #15219 Vega, #15186 Euclid) all disjoint — clean.
Origin Session ID: 75ed6708-c66b-4989-862d-2286e87abbf1
Retrieval Hint: "unit test chroma fixed port 18180 orphan wedge per-process derivation"