This ticket converts a concrete review-loop friction point into CI coverage. During the #11680 / #11681get_neighbors tool-shape follow-up, the operator called out the failure mode directly: if a change affects an MCP tool shape, the matching openapi.yaml must move with it or the MCP server may fail at boot/list-tools time. The requested shape is explicitly separate from the active implementation PR: add a new ticket for low-cognitive-load tests that boot/list Neo MCP servers and make CI go red when the tool surface is structurally broken.
The Problem
Agents and reviewers do not have a cheap, uniform answer to: "did this change break one of the Neo MCP server tool surfaces?" We can inspect diffs and individual service tests, but MCP exposure is assembled through per-server toolService.mjs + openapi.yaml wiring. A missing YAML field, invalid schema, stale path, or broken service mapping can survive local service-level tests and only surface when a harness asks the server for its tools.
Duplicate sweep result: #11110 is the closest prior artifact, but it is scoped to github-workflow after the M6 migration and explicitly leaves other MCP servers out of scope. This ticket is the broader cross-server smoke layer.
The Architectural Reality
V-B-A evidence gathered before filing:
Current MCP server entrypoints/specs exist for five servers: ai/mcp/server/neural-link, ai/mcp/server/knowledge-base, ai/mcp/server/github-workflow, ai/mcp/server/file-system, and ai/mcp/server/memory-core; each has mcp-server.mjs and openapi.yaml.
ai/mcp/server/BaseServer.mjs wires MCP tools/list to toolService.listTools({cursor, limit}) and maps each returned tool's name, inputSchema, outputSchema, and annotations into the MCP response.
Existing coverage is asymmetric: test/playwright/unit/ai/mcp/server/memory-core/McpServerToolLimits.spec.mjs calls listTools() for Memory Core, while test/playwright/unit/ai/mcp/server/github-workflow/ToolRegistration.spec.mjs performs static serviceMapping regex checks for a few tool names. There is no shared all-server listTools smoke harness.
#11110 documents the same bug class for github-workflow: boot-only checks can miss lazy OpenAPI/listTools failures; per-service tests do not exercise MCP wiring.
The Fix
Add a shared MCP smoke-test layer under test/playwright/unit/ai/mcp/server/ that enumerates the active Neo MCP servers and exercises their tool-listing surface.
Recommended implementation shape:
Add a reusable server fixture list for active MCP servers, including server name, mcp-server.mjs, toolService.mjs, and expected openapi.yaml path.
Add a fast unit smoke that imports each server's toolService.mjs, calls listTools(), and asserts:
no import/listTools error,
tool count is non-zero,
every tool has a valid name and inputSchema,
declared tool names are unique per server,
the generated tool list is compatible with the server's openapi.yaml operation IDs.
Where boot can be isolated without external daemons, add a stdio MCP handshake smoke per server that starts mcp-server.mjs, sends initialize, then tools/list, and fails cleanly on boot crash or malformed list response.
If a server cannot participate in stdio boot smoke without heavy dependencies, document the reason in the fixture and keep the fast toolService.listTools() guard mandatory.
Contract Ledger Matrix
Target Surface
Source of Authority
Proposed Behavior
Fallback
Docs
Evidence
MCP tools/list output for every active Neo MCP server
Unit CI exercises all active server tool-listing surfaces and fails on YAML parse, stale path, duplicate name, or mapping drift
Server-specific fixture note only when stdio boot requires unavailable external dependencies
Test names and comments should point to #11110 and this ticket
npm run test-unit -- test/playwright/unit/ai/mcp/server
Acceptance Criteria
A shared MCP server fixture enumerates all currently active Neo MCP servers: memory-core, knowledge-base, github-workflow, neural-link, and file-system.
Unit coverage calls listTools() for every fixture and asserts non-empty, well-formed, uniquely named tools.
Unit coverage verifies each server's listed tool names align with its openapi.yaml operation IDs, so YAML/toolService drift fails in CI.
At least one reusable stdio tools/list handshake helper exists; each server either uses it or documents a specific dependency reason why only the fast toolService smoke applies.
The new tests run through npm run test-unit -- test/playwright/unit/ai/mcp/server without requiring live Chroma, GitHub credentials, browser windows, or harness-local MCP config.
Existing targeted tests such as Memory Core tool-limit coverage and GitHub Workflow registration checks are either reused or left intact; this ticket does not delete narrower regression tests.
Out of Scope
Fixing any discovered MCP schema or server boot failure beyond the first small unblock needed to make the new smoke tests meaningful.
Changing MCP tool schemas, openapi.yaml descriptions, or service mappings except as required by a failing smoke test discovered during implementation.
Adding new MCP tools.
Running dockerized integration stacks as part of the fast unit smoke layer.
Avoided Traps / Gold Standards Rejected
Only test the recently touched tool. Rejected because the user-facing friction is cognitive load: reviewers do not always know which service methods become MCP tools. The listTools surface already knows.
Only boot the process. Rejected because #11110 documents that boot-only checks can miss lazy OpenAPI/listTools failures.
Full integration-only coverage. Rejected because the goal is a cheap CI warning. Dockerized integration remains valuable, but this guard should fail fast in unit CI.
One-off regex checks per server. Rejected because they reproduce the current asymmetry. A shared fixture prevents future servers from skipping the guard by accident.
Context
This ticket converts a concrete review-loop friction point into CI coverage. During the #11680 / #11681
get_neighborstool-shape follow-up, the operator called out the failure mode directly: if a change affects an MCP tool shape, the matchingopenapi.yamlmust move with it or the MCP server may fail at boot/list-tools time. The requested shape is explicitly separate from the active implementation PR: add a new ticket for low-cognitive-load tests that boot/list Neo MCP servers and make CI go red when the tool surface is structurally broken.The Problem
Agents and reviewers do not have a cheap, uniform answer to: "did this change break one of the Neo MCP server tool surfaces?" We can inspect diffs and individual service tests, but MCP exposure is assembled through per-server
toolService.mjs+openapi.yamlwiring. A missing YAML field, invalid schema, stale path, or broken service mapping can survive local service-level tests and only surface when a harness asks the server for its tools.Duplicate sweep result: #11110 is the closest prior artifact, but it is scoped to
github-workflowafter the M6 migration and explicitly leaves other MCP servers out of scope. This ticket is the broader cross-server smoke layer.The Architectural Reality
V-B-A evidence gathered before filing:
ai/mcp/server/neural-link,ai/mcp/server/knowledge-base,ai/mcp/server/github-workflow,ai/mcp/server/file-system, andai/mcp/server/memory-core; each hasmcp-server.mjsandopenapi.yaml.ai/mcp/server/BaseServer.mjswires MCPtools/listtotoolService.listTools({cursor, limit})and maps each returned tool'sname,inputSchema,outputSchema, and annotations into the MCP response.test/playwright/unit/ai/mcp/server/memory-core/McpServerToolLimits.spec.mjscallslistTools()for Memory Core, whiletest/playwright/unit/ai/mcp/server/github-workflow/ToolRegistration.spec.mjsperforms static serviceMapping regex checks for a few tool names. There is no shared all-server listTools smoke harness.github-workflow: boot-only checks can miss lazy OpenAPI/listTools failures; per-service tests do not exercise MCP wiring.The Fix
Add a shared MCP smoke-test layer under
test/playwright/unit/ai/mcp/server/that enumerates the active Neo MCP servers and exercises their tool-listing surface.Recommended implementation shape:
mcp-server.mjs,toolService.mjs, and expectedopenapi.yamlpath.toolService.mjs, callslistTools(), and asserts:nameandinputSchema,openapi.yamloperation IDs.mcp-server.mjs, sendsinitialize, thentools/list, and fails cleanly on boot crash or malformed list response.toolService.listTools()guard mandatory.Contract Ledger Matrix
tools/listoutput for every active Neo MCP serverai/mcp/server/*/openapi.yaml+ per-servertoolService.mjsnpm run test-unit -- test/playwright/unit/ai/mcp/serverAcceptance Criteria
listTools()for every fixture and asserts non-empty, well-formed, uniquely named tools.openapi.yamloperation IDs, so YAML/toolService drift fails in CI.tools/listhandshake helper exists; each server either uses it or documents a specific dependency reason why only the fast toolService smoke applies.npm run test-unit -- test/playwright/unit/ai/mcp/serverwithout requiring live Chroma, GitHub credentials, browser windows, or harness-local MCP config.Out of Scope
openapi.yamldescriptions, or service mappings except as required by a failing smoke test discovered during implementation.Avoided Traps / Gold Standards Rejected
Related
getNeighborsoutput schema follow-up that exposed the latest YAML/tool-shape review frictionOrigin Session ID: 019e44ba-d309-7e91-a819-36911fbf4e10 Retrieval Hint: "cross-server MCP listTools smoke", "boot each Neo MCP server", "openapi yaml toolService drift", "MCP tools/list CI guard"