LearnNewsExamplesServices
Frontmatter
id16171
titleRoute Playwright deployment snapshots to worker-local storage
stateClosed
labels
bugaitestingmodel-experience
assigneesneo-gpt-emmy
createdAtJul 30, 2026, 1:48 PM
updatedAtJul 30, 2026, 2:45 PM
githubUrlhttps://github.com/neomjs/neo/issues/16171
authorneo-gpt-emmy
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJul 30, 2026, 2:44 PM

Route Playwright deployment snapshots to worker-local storage

Closed Backlog/active-chunk-11 bugaitestingmodel-experience
neo-gpt-emmy
neo-gpt-emmy commented on Jul 30, 2026, 1:48 PM

Context

While validating PR #16170, the unfiltered cross-server unit smoke reached and passed the GitHub Workflow assertions, then failed in McpServerListToolsSmoke.spec.mjs with:

EPERM: operation not permitted, open '<canonical .neo-ai-data>/deployment-state/snapshot.json'

The failure was reproduced in a managed sandbox on 2026-07-30. Source inspection explains it:

  • test/playwright/unit/ai/mcp/server/McpServerListToolsSmoke.spec.mjs:511-570 reads the configured deployment snapshot, writes a two-service fixture to that same path, then restores or removes the prior bytes in finally.
  • ai/configBase.mjs:932 declares orchestrator.deploymentStateBridge.snapshotPath as the NEO_DEPLOYMENT_STATE_BRIDGE_SNAPSHOT_PATH leaf under the canonical plane root.
  • test/playwright/configTemplateResolver.mjs:108-139 already creates runner/worker-local storage and binds writable log, telemetry, heal-attempt, and recovery-run paths, but does not bind the deployment snapshot leaf.

The local producer regenerated a healthy four-service snapshot after the failed run, so no residual runtime corruption remains. The mutation window and cleanup failure are nevertheless real.

The Problem

A unit test currently treats a live plane member as disposable test storage.

Under a restricted runner, the fixture write can atomically replace the configured snapshot while the byte-for-byte restore is denied. Under an unrestricted runner, the finally restore has a second race: it can overwrite a newer snapshot emitted by the real producer after originalSnapshot was captured. Green cleanup therefore would not make the ownership shape safe.

This also makes a broad smoke suite fail for reasons unrelated to the operation being validated and forces maintainers to rerun a narrower grep just to separate the propositions.

The Architectural Reality

  • test/playwright/configTemplateResolver.mjs is the existing owner of Playwright runner/worker-local writable paths. It establishes NEO_TEST_STORAGE_ROOT and a distinct storageRoot per TEST_WORKER_INDEX before test dependencies evaluate.
  • NEO_DEPLOYMENT_STATE_BRIDGE_SNAPSHOT_PATH is already the declarative AiConfig env binding for this leaf. Supplying it at the Playwright bootstrap boundary preserves AiConfig as the resolved SSOT; mutating the shared singleton inside the test would violate ADR 0019 ยง4.
  • McpServerListToolsSmoke.spec.mjs should exercise the Knowledge Base and Memory Core public snapshot tools against one disposable fixture, not own production save/restore behavior.
  • test/playwright/unit/test/ConfigTemplateResolver.spec.mjs is the sibling contract suite for resolver scope and worker-local path behavior.
  • The Agent OS structure map confirms these existing owners; no new .mjs file or production service is required.

The Fix

  1. During activateStorageScope() scope entry, bind NEO_DEPLOYMENT_STATE_BRIDGE_SNAPSHOT_PATH beneath the current worker's disposable storageRoot.
  2. Pin the mapping and worker separation in ConfigTemplateResolver.spec.mjs.
  3. Remove canonical snapshot read/save/restore/remove behavior from the cross-server smoke. Write the fixture only to the resolved disposable path and let the existing worker-root cleanup own deletion.
  4. Preserve the Knowledge Base and Memory Core public-tool assertions against that fixture.
  5. Record a RED/GREEN witness showing the targeted test succeeds in the managed sandbox while the canonical snapshot's hash and mtime remain unchanged.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
Playwright deployment snapshot path configTemplateResolver.mjs + the existing AiConfig env leaf Resolve beneath <worker storageRoot>/deployment-state/snapshot.json before config evaluation Resolver-created temp root remains disposable and exit-cleaned Test JSDoc only Resolver path + distinct-worker assertions
KB/MC snapshot smoke McpServerListToolsSmoke.spec.mjs Both public tools read the same worker-local fixture Test fails visibly; no live-file restore path N/A Existing payload assertions + sandbox GREEN
Canonical deployment snapshot Production AiConfig plane member + producer Never read for backup, overwritten, restored, or removed by the unit No permission escalation or swallowed cleanup N/A Pre/post canonical hash and mtime unchanged

Decision Record impact

aligned-with ADR 0019. Isolation is constructed through the declared env-bound leaf before config evaluation; the shared AiConfig singleton is never mutated.

Acceptance Criteria

  • NEO_DEPLOYMENT_STATE_BRIDGE_SNAPSHOT_PATH resolves beneath each Playwright worker's NEO_TEST_STORAGE_ROOT.
  • ConfigTemplateResolver.spec.mjs pins the mapping and proves two worker indices receive distinct snapshot paths.
  • McpServerListToolsSmoke.spec.mjs contains no canonical snapshot backup/restore/remove lifecycle.
  • Knowledge Base and Memory Core snapshot/inspection assertions still consume one disposable fixture.
  • The focused cross-server smoke passes in a managed sandbox without permission escalation.
  • RED/GREEN evidence shows the canonical deployment snapshot hash and mtime are unchanged by the test.
  • The AiConfig test-mutation lint remains green; no shared singleton mutation or defensive config fallback is added.
  • If a workers:4 witness is practical in the focused suite, every worker reports a distinct snapshot path; otherwise the two-index resolver contract is the minimum gate.

Out of Scope

  • Changing the production deployment snapshot default, format, cadence, or public tool response.
  • Solving the broader order-dependent test pollution tracked by #15874.
  • Re-landing four CI workers, owned by #15861.
  • Refactoring other worker-local path bindings.
  • Permission escalation or a live-file save/restore workaround.

Avoided Traps

  • Mutate and restore AiConfig: violates ADR 0019 and remains order/race dependent.
  • Make the live file writable: removes the visible EPERM while preserving the stale-restore race.
  • Swallow cleanup failures: turns a runtime-plane mutation into a green test.
  • Keep a canonical backup copy: still races the real producer and makes test correctness depend on external state.

Related

Origin: #13926 / PR #13935. Worker-parallelism consumer: #15861. Broader isolation siblings: #15874 and #11976. Production canonical-root ownership remains with #15931 and is not duplicated here.

Live duplicate sweep: latest 20 open issues by creation time, recent 30 A2A messages across read states, exact GitHub issue/PR searches, Knowledge Base ticket search, and source history checked at 2026-07-30T11:47:07Z; no equivalent ticket or in-flight claim found.

Origin Session ID: 019fac4d-7844-7422-9486-7f73ccf308f5

Retrieval Hint: McpServerListToolsSmoke deployment snapshot EPERM worker-local configTemplateResolver

Authored by Emmy (GPT-5.6 Sol Ultra, Codex).

tobiu referenced in commit 761a6c8 - "fix(test): isolate deployment snapshots per worker (#16171) (#16172) on Jul 30, 2026, 2:44 PM
tobiu closed this issue on Jul 30, 2026, 2:44 PM