LearnNewsExamplesServices
Frontmatter
id11643
titleKB Ingestion Phase 5: Integration + Unit Test Parity with Memory Core
stateClosed
labels
epicaitestingarchitecture
assignees[]
createdAtMay 19, 2026, 2:07 PM
updatedAtJun 7, 2026, 7:13 PM
githubUrlhttps://github.com/neomjs/neo/issues/11643
authorneo-opus-ada
commentsCount3
parentIssue11624
subIssues
11644 Phase 5A — KB MCP Server Integration Test Parity with MC (pre-Phase-0/1 shippable)
11645 Phase 5B — KB Tenant Isolation Integration Tests (post-Phase-0/1D)
11646 Phase 5C — KB Unit Test Coverage Expansion (close 32-vs-12 parity gap with MC)
subIssuesCompleted3
subIssuesTotal3
blockedBy[]
blocking[]
closedAtMay 21, 2026, 2:16 PM

KB Ingestion Phase 5: Integration + Unit Test Parity with Memory Core

Closed v13.0.0/archive-v13-0-0-chunk-12 epicaitestingarchitecture
neo-opus-ada
neo-opus-ada commented on May 19, 2026, 2:07 PM

Context

Phase 5 sub-Epic of meta-Epic #11624 (Cloud-Native KB Ingestion for External Workspaces). NOT in Discussion #11623 §7 original; surfaced 2026-05-19 during operator-directed post-graduation review.

Empirical parity gap (V-B-A grounded; calibrated 2026-05-19T13:42Z per GPT epic-review):

  • Unit tests (path-scoped to test/playwright/unit/ai/services/):
    • memory-core/*.spec.mjs (incl. subtree): 23 specs
    • knowledge-base/*.spec.mjs (incl. subtree): 17 specs
    • ~74% parity (initial framing of 32 vs 12 was based on find -path "*memory-core*" which swept broader subtrees including ai/daemons/services/ — see #11646 for the path-scoped audit work)
  • Integration tests: 8 specs in test/playwright/integration/. KB has dual-server healthcheck.spec.mjs (per #10805 Lane A) AND partial SSE-transport query_documents coverage via KB_URL in RemoteMcpTransport.integration.spec.mjs (lines 4, 15, 21, 35). KB lacks dedicated equivalents of:
    • TeamPrivateRetrieval.integration.spec.mjs (memorySharing tenant scoping)
    • BackupRestoreWipe.integration.spec.mjs (manageDatabaseBackup integration coverage)
    • CrossTenantIsolation.integration.spec.mjs (cross-tenant identity isolation)
    • HeartbeatPropagation.integration.spec.mjs (full propagation; partial via healthcheck.spec.mjs)
    • AuthRejection.integration.spec.mjs (OIDC/proxy auth rejection paths)
    • OidcAuth.integration.spec.mjs (OIDC bearer authentication)
    • RemoteMcpTransport.integration.spec.mjs — Phase 5A should SPLIT/EXTEND the existing partial KB-SSE coverage rather than duplicate it (stdio path + tool-surface parity beyond query_documents remain uncovered)

Operator framing (2026-05-19): "we already have tickets for MC => pipeline, starting unified chroma, docker container, we want to test if MCP servers are accessible in there. mostly MC, and for KB we are lacking behind. meaning: this epic needs new tests here. and /unit-tests too."

The Problem

Cloud deployments of Agent OS require KB to operate in dockerized unified-Chroma topology with SSE + stdio transport, OIDC + proxy-header auth, tenant-isolated reads/writes, backup/restore, healthcheck observability. MC has comprehensive integration test coverage for these surfaces; KB has only healthcheck (dual-server dockerized verification).

Without parity coverage, Phase 0/1, 2, 3, 4 work risks regressions in operational substrate that's only caught at deployment time. Substrate-correct shape: test parity FIRST (Phase 5A pre-Phase-0/1 shippable), then evolve.

Sub-Decomposition

Three subs covering the parity surface, ordered by dependency:

  • Phase 5A — KB MCP Server Integration Test Parity (pre-Phase-0/1 shippable; independent — can start NOW)
  • Phase 5B — KB Tenant Isolation Integration Tests (blocked-by Phase 0/1D #11632)
  • Phase 5C — KB Unit Test Coverage Expansion (cross-cutting; can start any time)

Acceptance Criteria

  • All 3 sub-tickets filed with explicit cross-references back to this Epic
  • Phase 5A ships first (validates KB substrate floor BEFORE we evolve it)
  • Phase 5B integration tests run in CI gate; Phase 0/1 work cannot regress tenant-isolation guarantees
  • Phase 5C unit-test coverage approaches MC parity (target: 20+ KB unit specs from current 17 path-scoped to test/playwright/unit/ai/services/knowledge-base/; ≥85% of MC's 23 path-scoped to test/playwright/unit/ai/services/memory-core/)
  • Test fixtures (fixtures/composeWebServer.mjs, fixtures/mcpClient.mjs, util/assertSustainedHealth.mjs) reused across new KB tests — no parallel fixture infrastructure

Out of Scope

  • Cloud-ingestion-specific integration tests (synthetic external workspaces) → Phase 2F #11638 (use-case level; this Epic is infrastructure-parity level)
  • Performance benchmarking suite (correctness E2E only for V1)
  • ML-driven test generation

Avoided Traps

Trap Why rejected
Folding 5A into Phase 0/1 sub 5A validates pre-Phase-0/1 KB substrate; independent shipping path
Folding 5C into per-service unit tests 32-vs-12 parity gap is cross-cutting; needs explicit audit + backfill ticket, not silently distributed across feature work
Building parallel test fixture infrastructure composeWebServer.mjs + mcpClient.mjs already exist; KB tests REUSE per existing MC pattern
Skipping pre-Phase-0/1 test parity Substrate-evolution risks regressions in operational surfaces (auth, transport, healthcheck) without baseline coverage

Related

  • Parent meta-Epic: #11624
  • Reference ticket: #10805 (Lane A — dockerized healthcheck integration; precedent for unified topology testing)
  • Sibling phases: Phase 0/1 #11625, Phase 2 #11626, Phase 3 #11627, Phase 4 #11628
  • MC tests parallel'd: test/playwright/integration/*.integration.spec.mjs (7 specs lacking KB equivalent)

Origin Session ID

7360e917-1733-4cdd-a6f3-5ac51c34b838

Handoff Retrieval Hints

  • Existing dockerized fixture: test/playwright/integration/fixtures/composeWebServer.mjs
  • MCP test client: test/playwright/integration/fixtures/mcpClient.mjs
  • Health assertion helper: test/playwright/integration/util/assertSustainedHealth.mjs
  • Reference MC integration specs in test/playwright/integration/*.integration.spec.mjs — adapt each for KB MCP endpoint + KB-specific assertions (collection names, tool surface)
  • Memory anchor: feedback_npx_bypass_test_isolation.md — use npm run test-integration-unified, not npx playwright
  • Operator framing 2026-05-19: "this epic needs new tests here. and /unit-tests too." Phase 5 is the explicit-scope answer.