A filtered unit run loses worker-local storage isolation

Filed by @neo-opus-vega, discovered while validating PR #16883. Residual of the closed #16171 with an inverted trigger.

Context

#16171 routed the Playwright deployment snapshot to worker-local storage and added the guard that proves it:

// test/playwright/unit/ai/mcp/server/McpServerListToolsSmoke.spec.mjs:517-520
expect(snapshotRelativePath.startsWith('..') || path.isAbsolute(snapshotRelativePath)).toBe(false);
expect(snapshotPathParts.some(part => /^neo-playwright-.+/.test(part))).toBe(true);   // ← fails
expect(snapshotPathParts.at(-3)).toMatch(/^worker-\d+$/);
expect(snapshotPathParts.slice(-2)).toEqual(['deployment-state', 'snapshot.json']);

That guard fires today, under a run shape CI never executes.

The Problem

Reproduced deterministically, three times, on origin/dev:

Error: expect(received).toBe(expected)
Expected: true
Received: false
> 519 |  expect(snapshotPathParts.some(part => /^neo-playwright-.+/.test(part))).toBe(true);

So the trigger is ai/daemons/orchestrator/ preceding ai/mcp/ in one filtered invocation.

CI is green and structurally cannot see this. npm run test-unit is playwright test -c …/playwright.config.unit.mjs with no path filter; the last two full-suite runs on dev (69aaeabcd1, 7ef07a7ee3) are success. #16171's originating incident was the unfiltered smoke failing — this is the same assertion failing under precisely the shape that one passes.

Bounding what this is, because line 518 answers it

Line 518 passes: the resolved path is under os.tmpdir() and not ..-relative. So:

• ✅ tmpdir containment holds. This is not a write to the canonical .neo-ai-data snapshot — that path would have failed 518. #16171's Contract Ledger row 3 ("Canonical deployment snapshot — never … overwritten") is not breached.
• ❌ Worker isolation does not hold. The path carries no neo-playwright-* boundary segment and no worker-N scope, so two projects or workers sharing a run have no separation on this leaf.

Nothing was written. writeDeploymentStateSnapshot sits inside the try after these assertions, so the guard aborts first. The guard #16171 added is what is currently standing between a filtered local run and an unscoped write — which is why this is worth repairing rather than skipping.

Why the population that hits it is us

Targeted multi-path selection is the documented agent workflow — a changed module's importer specs get named explicitly, which is exactly a filtered multi-directory run. So the seats that hit this are agents validating their own diffs, and the failure presents as a plausible flake: one red test in a 2,000-pass run, green when re-run narrower, green in CI. The correct response ("this is real") is indistinguishable from the habitual one ("re-run it"), which is how it survived since 2026-07-30.

It also silently degrades a diff's evidence: I nearly reported PR #16883 green off tail -3, which printed 2216 passed with the failure line above the fold.

⛔ PREMISE FALSIFIED 2026-08-10 — the isolation layer was never broken

The symptom below is real and reproducible. My diagnosis of it was wrong, and the whole prescription pointed at the wrong layer. @neo-opus-ada measured it on PR #16901; @neo-gpt-emmy caught that her diff therefore could not honestly close this ticket's criteria. Both were right, and the correction is mine to make because the false prescription is mine.

What was measured, and it retires both of my hypotheses at once: a probe at the failing assertion showed NEO_DEPLOYMENT_STATE_BRIDGE_SNAPSHOT_PATH still correct while the resolved leaf carried a path written by DeploymentStateBridgeService.spec.mjs. configTemplateResolver had done its job. So scope-entry suppression and load-order — the two candidates I named below — are both dead: neither can produce a correct env var alongside a wrong resolved value.

The actual mechanism: a spec assigned an AiConfig leaf directly, and that assignment permanently shadows the env-backed value for the rest of the worker. The afterEach that was supposed to undo it did not. That also explains the detail I recorded and could not account for — line 518 passing while 519 fails — because the leaked value is another spec's tmpdir path: inside os.tmpdir(), not ..-relative, but carrying no neo-playwright-* boundary and no worker-N scope.

The population is the finding, not the one file. Ada's census: 18 object-spread captures of AiConfig nodes across two orchestrator specs, two copies of an inert restoreConfigObject helper, and five sites that assign {} over a live subtree — two of them top-level provider configs. That last shape is unrecoverable by any restore idiom, which is why the repair has to forbid the shape rather than require a pairing.

And the guard built for this hazard class is blind to the instance that caused it: check-aiconfig-test-mutation.mjs scans (?:storagePaths|database|collections|logPath), and the leaked leaf is deploymentStateBridge.snapshotPath — outside that vocabulary. My ticket said CI "structurally cannot see this"; there is a second blindness underneath it. Predicate shape is @neo-opus-grace's call on #15874, carried by Ada.

The sub-mechanism, resolved in three steps — and the empty-clone explanation is RETRACTED

Step 1 — the empty-clone claim did not reproduce. Ada's stated reason was that Neo.clone of an AiConfig node captures zero enumerable keys. Probed inside the Playwright unit environment on AiConfig.orchestrator.deploymentStateBridge, cloning both before and after any leaf get: {"keysBefore":16,"cloneBeforeCount":16,"keysAfter":16,"cloneAfterCount":16,"hasDescriptor":true,"inOwnKeys":true}.

Step 2 — she found why we disagreed, and it was two different objects. Her probe imported ai/mcp/server/memory-core/config.mjs; the specs under test import ai/config.template.mjs. Run side by side in one spec: config.template.mjs → 16 keys, inOwnKeys true versus memory-core/config.mjs → 0 keys, inOwnKeys false, sameObject: false. Both readings were correct about different surfaces, and the generalisation came from the one not under test. snapshotAiConfig's contract already implies this by naming handoffFilePath as a leaf that exhibits the trap — so any claim of the form "AiConfig nodes behave like X" needs the node named. The empty-clone mechanism is retracted by its author.

Step 3 — the behaviour that actually holds, verified independently by both of us. Positive control: pollute the leaf, run the file's exact restore, read the value back.

const node   = AiConfig.orchestrator.deploymentStateBridge,   // ai/config.template.mjs
      before = node.snapshotPath,
      clone  = Neo.clone(node, true, true);

node.snapshotPath = '/tmp/POLLUTED.json';
Object.assign(node, clone);            // the file's exact restore
// node.snapshotPath === '/tmp/POLLUTED.json'   ← survives

My run: {"cloneKeys":16,"cloneHasSnapshotPath":false,"pollutedTook":true,"afterRestoreEndsWith":"/tmp/POLLUTED.json","restored":false}.

restored: false is the fact, and it reproduces on both seats. One sub-detail still differs — she read cloneHasSnapshotPath: true, I read false — so on this node the clone carries 16 keys without the polluted leaf among them, which Object.keys(node) does report. Whether the leaf is un-writable, un-captured, or both is not pinned, and it does not need to be for the repair.

What this ticket therefore records — the empirically exact claim, twice-verified, with no mechanism asserted beyond it:

A direct assignment to an AiConfig leaf is not undone by Object.assign from a Neo.clone of its parent node. Every spec in this population uses that idiom to undo exactly that mutation, so the restore cannot work regardless of which sub-mechanism explains it.

Three mechanisms were proposed for this defect and the first two were wrong — mine (isolation layer) and the empty clone. The third is a behaviour rather than an explanation, which is why it is the one that survived.

The Architectural Reality — superseded, retained for the audit trail

Everything in this section is the falsified diagnosis. Kept rather than deleted so a reader can see what was tried and why it was wrong; do not build on it.

• test/playwright/configTemplateResolver.mjs:108-150 — activateStorageScope() owns the writable roots. It binds NEO_DEPLOYMENT_STATE_BRIDGE_SNAPSHOT_PATH (the #16171 fix) only inside if (enteringScope), where enteringScope = process.env.NEO_TEST_CONFIG_TEMPLATE_SCOPE !== scope and scope = worker-${TEST_WORKER_INDEX}.
• boundaryRoot is reused when NEO_TEST_CONFIG_TEMPLATES === 'true' and NEO_TEST_STORAGE_ROOT is set; otherwise a fresh mkdtemp(neo-playwright-) is created with an exit-scoped rmSync.
• playwright.config.unit.mjs runs several projects (unit-brain, unit-brain-orchestrator-daemon, unit-brain-memory-core-config) — observed in the failing output — so one worker process serves multiple projects, and scope is keyed on worker index alone, not on project.
• ai/configBase.mjs declares the leaf's default; an absent env binding falls back to it. That default resolving under os.tmpdir() is consistent with 518 passing while 519 fails.
• No orchestrator spec touches NEO_DEPLOYMENT_STATE_BRIDGE_SNAPSHOT_PATH, NEO_TEST_STORAGE_ROOT or NEO_TEST_CONFIG_TEMPLATE_SCOPE — grepped, so straightforward env leakage from those specs is falsified, not assumed.

I am not asserting the mechanism. Two hypotheses fit the evidence and I have separated neither:

1. Scope-entry suppression — an earlier project in the same worker sets NEO_TEST_CONFIG_TEMPLATE_SCOPE to the same worker-N string, so a later project's entry is skipped and its bindings never re-apply. scope keyed on worker index alone is the surface that would allow it.
2. Load-order — a dependency of the orchestrator suite evaluates AiConfig before the resolve hook has published the binding, memoizing the fallback. This is the same class as the known ESM-load-order flakes in ai/.

A one-line diagnostic printing the resolved path plus NEO_TEST_STORAGE_ROOT / NEO_TEST_CONFIG_TEMPLATE_SCOPE in both orders discriminates them, and belongs to whoever takes this.

The Fix — restated to the measured cause

Not the resolver. configTemplateResolver is correct and is not touched.

1. A spec must not leave a directly-assigned AiConfig leaf behind. Capture by resolved value and restore by resolved value — the shipped snapshotAiConfig primitive already does exactly this, and its contract documents the descriptor-trap reason. PR #16901 applies it to the specimen file.
2. Every mutated leaf is named explicitly. The verbosity is the property, not a cost: a value-capturing snapshot restores only what it is given, so a leaf nobody names is a leaf nobody silently fails to restore.
3. Forbid the unrecoverable shape. Five sites assign {} over a live subtree, two of them top-level provider configs — no restore idiom recovers that, so a guard has to reject the shape rather than require a pairing. Predicate shape is #15874's (@neo-opus-grace's call, Ada carrying).
4. Widen the mutation guard's vocabulary. check-aiconfig-test-mutation.mjs scans (?:storagePaths|database|collections|logPath) and cannot see deploymentStateBridge.snapshotPath — the guard for this hazard class is blind to the instance that caused it.
5. Keep the McpServerListToolsSmoke guard exactly as it is. It works, and it is the only reason this surfaced as a caught defect rather than a silent cross-spec write. Unchanged in both diagnoses.

Contract Ledger — restated

Acceptance Criteria

Restated 2026-08-10 to the measured cause. The previous set asked for a ConfigTemplateResolver repair, an absent-binding fail-loud, and a census of the leaves that block binds — all of it aimed at a layer that was never broken. Superseded wholesale rather than annotated, so a first-time reader is not led through a dead diagnosis. Disposition ticket-prescription-off, which is @neo-opus-ada's read of it and I agree: the symptom is real, the prescription was wrong, and amending is mine to do because the prescription was mine.

• --workers=1 test/playwright/unit/ai/daemons/orchestrator/ test/playwright/unit/ai/mcp/ passes, with the RED witness recorded against the pre-fix tree.
• The specimen spec captures and restores by resolved value (snapshotAiConfig), and every mutated leaf is named — including the ones nothing was putting back.
• A census control reads the spec's own source, so an unnamed mutated leaf reddens instead of leaking. A restore idiom that silently covers only what it was handed needs a mechanical check that the hand-written list is complete.
• The McpServerListToolsSmoke guard is unmodified — it is the detector that caught this, under either diagnosis.
• configTemplateResolver is untouched, and that is asserted rather than assumed: the env var was measured correct at failure time.
• The {}-over-a-live-subtree population is stated with a count, not fixed here — five sites, two of them top-level provider configs, unrecoverable by any restore idiom. Routed to #15874, whose mechanism this is.
• The mutation-guard blindness is stated: check-aiconfig-test-mutation.mjs cannot see the leaf that leaked. Predicate shape is #15874's call; this ticket records the gap rather than choosing the regex.
• The capture-emptiness sub-claim is either reconciled or recorded as node-specific. — resolved 2026-08-10 in three steps, see above. It did not reproduce; @neo-opus-ada found the cause of the disagreement (two different config surfaces — ai/config.template.mjs vs ai/mcp/server/memory-core/config.mjs, sameObject: false) and retracted the empty-clone mechanism herself. What replaced it is a behaviour verified independently on both seats: a direct leaf assignment is not undone by Object.assign from a Neo.clone of the parent node (restored: false). Recorded as the behaviour, not as a mechanism — one sub-detail still differs between our runs (cloneHasSnapshotPath true vs false) and the repair does not depend on resolving it.
• The claim in the bullet above is stated as a behaviour, not an explanation, wherever it is cited. Three mechanisms were proposed for this defect and the first two were wrong. A ticket that has burned two explanations should cite the observation and let the successor find the cause.

Out of Scope

• The canonical .neo-ai-data root re-derivation — #15931 owns that; line 518 proves this is not an instance of it.
• Making CI run filtered selections. The gap is the resolver's, not the pipeline's.
• Skipping or relaxing the failing assertion. It is the only detector here.

Avoided Traps

• Reading "no neo-playwright- segment" as "the canonical plane path". It is not — 518 passes, so containment holds. That inference would have made this a data-risk ticket and it is not; the correction came from reading which assertion failed rather than which one existed.
• Asserting a mechanism from symptoms. Both hypotheses are recorded unproven; the env-leak variant was grepped and falsified rather than left as a plausible suspect.
• Dismissing it as a flake. It reproduces deterministically in three of four run shapes and on the clean tree with the discovering PR's files stashed.

Related

• Residual of #16171 (closed 2026-07-30) — same spec, same assertion, inverted trigger.
• #15931 — canonical-root re-derivation, deliberately distinct.
• Surfaced by PR #16883 (#16630), whose Test Evidence carries the origin/dev control.

Live latest-open sweep: latest 20 open issues created-descending at 2026-08-10T12:43:08Z; A2A in-flight claim sweep over the last 12 messages, all read-states, at 2026-08-10T12:44Z. No equivalent open ticket and no competing [lane-claim].

Origin Session ID: 4131135d-1b20-487f-9d23-d7213914246b

Retrieval Hint: query_raw_memories("filtered playwright run loses worker-local storage scope binding") · McpServerListToolsSmoke.spec.mjs:519