LearnNewsExamplesServices
Frontmatter
id11644
titlePhase 5A — KB MCP Server Integration Test Parity with MC (pre-Phase-0/1 shippable)
stateClosed
labels
enhancementaitesting
assigneesneo-gpt
createdAtMay 19, 2026, 2:08 PM
updatedAtJun 7, 2026, 7:13 PM
githubUrlhttps://github.com/neomjs/neo/issues/11644
authorneo-opus-ada
commentsCount0
parentIssue11643
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtMay 19, 2026, 5:00 PM

Phase 5A — KB MCP Server Integration Test Parity with MC (pre-Phase-0/1 shippable)

Closed v13.0.0/archive-v13-0-0-chunk-12 enhancementaitesting
neo-opus-ada
neo-opus-ada commented on May 19, 2026, 2:08 PM

Context

Sub of Phase 5 Epic #11643 (meta-Epic #11624).

Pre-Phase-0/1 shippable — INDEPENDENT, can start NOW. Validates KB MCP server foundation BEFORE Phase 0/1 evolves it. Substrate-correct safety net for the rest of the meta-Epic.

Topology anchor: Per ADR 0003 — Chroma Topology Unified Only, tests target the unified-Chroma topology: one ChromaDB daemon shared by KB + MC MCP servers (KB-owned knowledge-base collection vs MC-owned neo-agent-memory + neo-agent-sessions). Existing composeWebServer.mjs fixture already wraps unified-topology docker-compose.

The Problem

Calibrated 2026-05-19T13:42Z (GPT epic-review on #11643): my initial framing of "zero KB references in RemoteMcpTransport.integration.spec.mjs" was incomplete V-B-A. That spec actually does call query_documents against KB_URL for transport tests (lines 4, 15, 21, 35) — KB has partial SSE-transport coverage already. So 5A's task is to SPLIT/EXTEND that surface, not duplicate it.

KB's healthcheck.spec.mjs (#10805 Lane A) gives dual-server dockerized healthcheck. RemoteMcpTransport.integration.spec.mjs gives partial SSE-transport query_documents coverage via KB_URL. The remaining MC integration specs lack KB equivalents:

  • RemoteMcpTransport.integration.spec.mjs — SSE + stdio transport (partial KB coverage exists via query_documents; needs SPLIT/EXTEND, not duplicate — stdio path + tool-surface parity beyond query_documents + ask_knowledge_base not yet covered)
  • AuthRejection.integration.spec.mjs — OIDC/proxy auth rejection
  • OidcAuth.integration.spec.mjs — OIDC bearer authentication
  • BackupRestoreWipe.integration.spec.mjs — backup/restore/wipe via JSONL
  • HeartbeatPropagation.integration.spec.mjs — healthcheck propagation in unified topology

KB has equivalent surfaces (manageDatabaseBackup action: 'export'/'import'/'truncate' lines 236-409 of DatabaseService.mjs; KB MCP server via SSE/stdio; OIDC + proxy-header auth via shared AuthService) but no dedicated KB-side integration test coverage. Without it, Phase 0/1 work risks silently regressing these operational surfaces.

The Fix

Create parallel KB integration test specs at test/playwright/integration/:

  1. KBRemoteMcpTransport.integration.spec.mjs — KB MCP server stdio accessibility (NOT yet covered) + tool-surface parity beyond query_documents (e.g. ask_knowledge_base, manage_knowledge_base, get_class_hierarchy, get_document_by_id, list_documents); reuse the KB_URL partial SSE coverage already in RemoteMcpTransport.integration.spec.mjs as the precedent and extend rather than duplicate
  2. KBAuthRejection.integration.spec.mjs — KB MCP server rejects unauthenticated requests; rejects wrong-identity OIDC tokens; rejects untrusted proxy-header injection
  3. KBOidcAuth.integration.spec.mjs — KB MCP server with OIDC bearer authentication; identity stamped server-side; access to KB tools (manage_knowledge_base, ask_knowledge_base, query_documents, etc.) gated correctly
  4. KBBackupRestoreWipe.integration.spec.mjs — KB manageDatabaseBackup action: 'export' produces JSONL; action: 'import' restores embeddings without re-embedding; action: 'truncate' clears with confirmation gate; mode: 'replace' / 'merge' semantics
  5. KBHeartbeatPropagation.integration.spec.mjs — KB healthcheck under sustained-liveness probe; KB collection state in unified topology healthcheck.database.connection.collections.knowledgeBase block

Each test reuses existing fixtures:

  • fixtures/composeWebServer.mjs (dockerized server wrapper)
  • fixtures/mcpClient.mjs (MCP tool dispatch client)
  • util/assertSustainedHealth.mjs (health-probe helper)

Acceptance Criteria

  • KBRemoteMcpTransport.integration.spec.mjs exists and passes; covers SSE + stdio
  • KBAuthRejection.integration.spec.mjs exists and passes; covers 3 rejection paths
  • KBOidcAuth.integration.spec.mjs exists and passes; covers OIDC happy path + identity stamping
  • KBBackupRestoreWipe.integration.spec.mjs exists and passes; covers export/import/truncate via JSONL
  • KBHeartbeatPropagation.integration.spec.mjs exists and passes; covers sustained-liveness + collection state
  • All tests run via npm run test-integration-unified (NOT npx playwright; per memory anchor on test isolation)
  • All tests skip cleanly when docker unavailable (test.skip(readiness.dockerAvailable === false, ...) pattern from healthcheck.spec.mjs)
  • CI integration: failures gate Phase 0/1 work
  • No parallel fixture infrastructure created; reuse existing 3 fixture/util helpers

Out of Scope

  • Phase 0/1 contracts integration tests → Phase 5B (post-Phase-0/1D)
  • Cloud-ingestion-specific E2E tests → Phase 2F (#11638)
  • Performance benchmarking
  • Custom-source/parser registration tests → covered by Phase 0/1B (#11630) unit tests + Phase 5C cross-cutting coverage

Related

  • Parent: #11643
  • Reference precedent: #10805 Lane A (healthcheck.spec.mjs pattern); MC integration specs are the parallel templates
  • Existing fixtures: test/playwright/integration/fixtures/composeWebServer.mjs, mcpClient.mjs, util/assertSustainedHealth.mjs

Origin Session ID

7360e917-1733-4cdd-a6f3-5ac51c34b838

Handoff Retrieval Hints

  • Adapt each MC integration spec for KB endpoint + KB tool surface. The dispatch pattern is identical (MCP JSON-RPC); only endpoints + tool names + assertion targets differ.
  • Start with KBBackupRestoreWipe.integration.spec.mjsmanageDatabaseBackup already exists; lowest-implementation-risk for first parity test
  • KBAuthRejection + KBOidcAuth share auth substrate with MC equivalents via shared AuthService — most code is reusable
tobiu referenced in commit 4359a77 - "test(kb): add KB integration parity coverage (#11644) (#11648) on May 19, 2026, 5:00 PM
tobiu closed this issue on May 19, 2026, 5:00 PM