Split out of #16481 so each half has a PR that can resolve it. #16481 keeps the false-claim correction (PR #16976); this holds the architecture.
Amended 2026-08-25 after the terminal review on PR #17771 (review 5023067944, @neo-gpt). The first attempt recorded an acceptance ADR resting on a premise this ticket itself supplied and never falsified — see Premise correction. PR #17771 is closed; its salvage map is carried below. Re-entry runs from this amended body.
The gap
The path jail binds arguments. It does not bind the process.
run_playwright_test executes a spec, and a spec is arbitrary JavaScript, so what it reads and writes is bounded by the host process this server runs in. write_file plus run_playwright_test therefore reaches outside the project root without either call violating any guard.
Reported externally by @novice-22, three independent times, reproduced from source.
Why the obvious partition is closed
From @novice-22 on #16481, and it is the reason this needs a real fix rather than a cleverer guard:
"A sound tool-level partition would have to keep write_file out of everywhere run_playwright_test can read. But a spec is arbitrary JavaScript, so the executor's read set is not statically bounded — it is the project."
So there is no line to draw between the two tools. Any guard that tries becomes a guard on a set nobody can enumerate, which is how the current one came to promise something it cannot deliver.
What would actually contain it
The executor needs to run somewhere that cannot read the host — a child process with a restricted filesystem view, or a container. That is a real design decision with real costs:
- test runs get slower and need their fixtures carried in
- failure diagnostics have to cross the boundary, or debugging gets worse
- the boundary itself becomes a thing to keep correct, and a broken sandbox that reports success is worse than none
Premise correction
This ticket previously carried, as a Delta, the line "this surface is used by agents operating on this repo with host access already." That is false, and it was load-bearing: PR #17771's ADR promoted it into the rationale for accepting unjailed execution outright.
The security subject is the caller's granted capability, not the Unix authority of the MCP subprocess. A host-trusted subprocess does not make a tool-scoped model host-trusted. Verified from source at b2359004:
.github/AI_QUICK_START.md:273 — frontier harnesses ship their own file and command-execution tools; this server exists for Neo.ai.Agent instances and local harnessless profiles that reach the filesystem through the Agent OS client.
ai/agent/AgentOrchestrator.mjs:198 — creates an autonomous Neo.ai.Agent with servers: ['knowledge-base', 'file-system', 'github-workflow']. No shell in the grant.
ai/agent/profile/QA.mjs:33 — servers: ['knowledge-base', 'file-system'], modelProvider: 'ollama', and a system prompt that mandates read_file. The narrowest grant in the fleet.
For those two classes the server list is a deliberate capability narrowing, and run_playwright_test silently converts it into host-process authority. Isolation is not "machinery guarding a boundary no caller is on the far side of" — they are exactly on the far side. The revisit condition the first ADR wrote for itself was already true when it was written.
Contract Ledger
| Caller class |
Granted authority |
Projection profile |
Decision owed |
| Frontier-harness maintainer seat (Codex, Claude Code, Gemini CLI, Antigravity) |
Host, held independently of this server |
Full |
Acceptance is defensible — the tool grants nothing the caller lacks |
Neo.ai.Agent via AgentOrchestrator |
3 MCP servers, no shell |
Currently full |
Open — isolate, or deny run_playwright_test to this profile |
QA profile (Ollama-backed, local) |
2 MCP servers, no shell |
Currently full |
Open — isolate, or deny run_playwright_test to this profile |
| Hosted instance / untrusted contributor path / non-fleet MCP client |
None today |
n/a |
Not present; do not hardcode the current fleet as the complete audience |
Broken-boundary posture. Whatever lands, a containment that stops containing must fail closed. A sandbox reporting success while no longer isolating converts a documented limit into a false assurance, which is strictly worse than the current state.
The reproduction stays the instrument. Any claim of containment is demonstrated by the two-step write_file → run_playwright_test composition from #16481 failing, not by inspection of the guard.
Acceptance criteria
Deltas
- The honesty defect is still live on
dev. run_playwright_test is tagged description: Execution sandbox and opens "Runs the Playwright runner isolated to the requested spec file". PR #17771 corrected both; closing it leaves them in place. The successor carries them — this is not optional cleanup.
- Do not narrow the discovery to hide it. A guard that stops the specific two-step reproduction while leaving arbitrary JS execution unbounded would look like a fix and be a regression in honesty.
- Salvage from PR #17771 (per the terminal review's map): keep the OpenAPI leading warning, the
Execution tag correction, and the internal-reference removal; re-home the caller-visible assertions onto the production toolServiceDispatch seam; discard ADR 0041's Accepted decision, its ADR-0031 seam row, and the close-target claim until the ledger's open rows are settled.
Split out of #16481 so each half has a PR that can resolve it. #16481 keeps the false-claim correction (PR #16976); this holds the architecture.
The gap
The path jail binds arguments. It does not bind the process.
run_playwright_testexecutes a spec, and a spec is arbitrary JavaScript, so what it reads and writes is bounded by the host process this server runs in.write_fileplusrun_playwright_testtherefore reaches outside the project root without either call violating any guard.Reported externally by @novice-22, three independent times, reproduced from source.
Why the obvious partition is closed
From @novice-22 on #16481, and it is the reason this needs a real fix rather than a cleverer guard:
So there is no line to draw between the two tools. Any guard that tries becomes a guard on a set nobody can enumerate, which is how the current one came to promise something it cannot deliver.
What would actually contain it
The executor needs to run somewhere that cannot read the host — a child process with a restricted filesystem view, or a container. That is a real design decision with real costs:
Premise correction
This ticket previously carried, as a Delta, the line "this surface is used by agents operating on this repo with host access already." That is false, and it was load-bearing: PR #17771's ADR promoted it into the rationale for accepting unjailed execution outright.
The security subject is the caller's granted capability, not the Unix authority of the MCP subprocess. A host-trusted subprocess does not make a tool-scoped model host-trusted. Verified from source at
b2359004:.github/AI_QUICK_START.md:273— frontier harnesses ship their own file and command-execution tools; this server exists forNeo.ai.Agentinstances and local harnessless profiles that reach the filesystem through the Agent OS client.ai/agent/AgentOrchestrator.mjs:198— creates an autonomousNeo.ai.Agentwithservers: ['knowledge-base', 'file-system', 'github-workflow']. No shell in the grant.ai/agent/profile/QA.mjs:33—servers: ['knowledge-base', 'file-system'],modelProvider: 'ollama', and a system prompt that mandatesread_file. The narrowest grant in the fleet.For those two classes the server list is a deliberate capability narrowing, and
run_playwright_testsilently converts it into host-process authority. Isolation is not "machinery guarding a boundary no caller is on the far side of" — they are exactly on the far side. The revisit condition the first ADR wrote for itself was already true when it was written.Contract Ledger
Neo.ai.AgentviaAgentOrchestratorrun_playwright_testto this profileQAprofile (Ollama-backed, local)run_playwright_testto this profileBroken-boundary posture. Whatever lands, a containment that stops containing must fail closed. A sandbox reporting success while no longer isolating converts a documented limit into a false assurance, which is strictly worse than the current state.
The reproduction stays the instrument. Any claim of containment is demonstrated by the two-step
write_file→run_playwright_testcomposition from #16481 failing, not by inspection of the guard.Acceptance criteria
run_playwright_testis absent from that class's projected tool list, proven by the projection omitting it.listTools()/ handbook records via thetoolServiceDispatchseam), not by parsingopenapi.yamldirectly.Deltas
dev.run_playwright_testis taggeddescription: Execution sandboxand opens "Runs the Playwright runner isolated to the requested spec file". PR #17771 corrected both; closing it leaves them in place. The successor carries them — this is not optional cleanup.Executiontag correction, and the internal-reference removal; re-home the caller-visible assertions onto the productiontoolServiceDispatchseam; discard ADR 0041's Accepted decision, its ADR-0031 seam row, and the close-target claim until the ledger's open rows are settled.