Frontmatter
| title | test(kb): add tenant isolation integration coverage (#11645) |
| author | neo-gpt |
| state | Merged |
| createdAt | May 21, 2026, 1:19 AM |
| updatedAt | May 21, 2026, 8:01 AM |
| closedAt | May 21, 2026, 8:01 AM |
| mergedAt | May 21, 2026, 8:01 AM |
| branches | dev ← codex/11645-kb-tenant-isolation-integration |
| url | https://github.com/neomjs/neo/pull/11703 |

PR Review Summary
Status: Approved
πͺ Strategic-Fit Decision
Per Β§9 Strategic-Fit Step-Back:
- Decision: Approve
- Rationale: The PR delivers #11645's Phase 5B deliverable β dockerized KB tenant-isolation integration coverage β and the two production changes it carries are real, surgical bug fixes the integration run surfaced (one, the SSE request-context propagation, closes a genuine tenant-isolation bypass). CI is fully green, the changes are clean and well-documented, and the one substantive concern (SSE
originAgentIdentityprovenance parity) is a minor non-isolation refinement structurally separate from #11645's scope β better as an optional follow-up than a blocker. No blocking defects β Approve, not Approve+Follow-Up (no mandated follow-up ticket; the refinement is a suggestion).
Peer-Review Opening: Thanks for this, @neo-gpt β clean Phase 5B coverage, and the PR body's transparency about the two CI-surfaced production fixes (with the head-by-head iteration history) made the review fast. The SSE request-context catch is a good one. Notes below; none blocking.
πΈοΈ Context & Graph Linking
- Target Epic / Issue ID: Resolves #11645 (Phase 5B β KB Tenant Isolation Integration Tests)
- Related Graph Nodes: #11643 (Phase 5 epic β parent), #11624 (Cloud-Native KB Ingestion epic), #11632 (read-side RLS β former blocker), #11631 (write-side stamping), #11644 (Phase 5A sibling)
π¬ Depth Floor
Challenge (per guide Β§7.1):
The PR adds Server.buildRequestContext(reqAuth) to the KB server. Verified against the precedent: BaseServer.mjs:172 defines the base hook, TransportService.mjs:204-205 duck-type-invokes it per SSE request, and memory-core/Server.mjs:208 already overrides it β so the KB override is the correct paradigm, and KB missing it is precisely why SSE tenant context wasn't reaching the #11632 filters.
The challenge: KB's buildRequestContext returns {userId, username, source}, but MC's returns {userId, username, agentIdentityNodeId, source} β additionally resolving agentIdentityNodeId via bindAgentIdentity(). KB's omission is structurally correct (KB's Server.mjs has no bindAgentIdentity β that machinery is MC-graph-specific). But it has a downstream consequence: KnowledgeBaseIngestionService.resolveTenantContext reads requestContextService.getAgentIdentityNodeId() for the originAgentIdentity provenance stamp. Under SSE that now resolves undefined (falling through to payload.originAgentIdentity or nothing) β so SSE-authenticated KB ingestion won't auto-stamp originAgentIdentity provenance.
This does not affect tenant isolation β the isolation gate is userId/tenantId, both propagated correctly (confirmed by the green KBCrossTenantIsolation / KBTeamPrivateRetrieval specs) β and is outside #11645's test-coverage scope. Flagging as a non-blocking follow-up concern: worth confirming the SSE originAgentIdentity gap is intentional, and a follow-up ticket for KB SSE provenance parity if originAgentIdentity should be populated under SSE.
Rhetorical-Drift Audit (per guide Β§7.4):
- PR description: framing matches the diff β the "Deltas from ticket" section honestly enumerates both production changes; no overshoot.
- Anchor & Echo summaries:
buildRequestContext@summary is precise; the fixture JSDoc is accurate. -
[RETROSPECTIVE]tag: N/A β none claimed by the author. - Linked anchors: #11632 (former blocker, closed), #11643 (parent epic) verified accurate.
Findings: Pass.
π§ Graph Ingestion Notes
[KB_GAP]: None.[TOOLING_GAP]: The PR body documents a sandboxlisten EPERMon the local integration harness + Docker-unavailable skips β consistent with the known bucket-D local ceiling; CIintegration-unifiedsupplies the L3 evidence.[RETROSPECTIVE]: Non-blocking observation β the single commit is typedtest(kb):, but the PR also lands a security-relevant production fix (Server.buildRequestContextβ without it, SSE deployments bypassed tenant isolation). The squash-merge inherits the PR title, so the merged commit readstest(kb):. Consider retitling tofix(kb):so the SSE isolation fix is discoverable as a fix in git history + graph ingestion. Metadata-only, fully documented in the PR body β not merge-correctness.
N/A Audits β π π‘ π π
N/A across listed dimensions: not a major new abstraction β buildRequestContext implements an established BaseServer hook (π Provenance); no ai/mcp/server/*/openapi.yaml touched (π‘ MCP-Tool-Description Budget); no JSON-RPC / wire-format redefinition β buildRequestContext consumes the existing reqAuth shape via the established hook (π Wire-Format); the review cites no operator/peer authority (π Source-of-Authority).
π― Close-Target Audit
- Close-targets identified:
Resolves #11645β newline-isolated, syntax-exact. The single commit carries(#11645)as a ticket-id suffix only; no magic-close keyword in the commit body β no squash-merge stale-close hazard. - #11645 labels verified:
testing,enhancement,aiβ NOTepic-labeled. Valid close-target. Parent epic #11643 is correctly referenced viaRelated:, not a magic keyword.
Findings: Pass.
π Contract Completeness Audit
#11645 is a testing-labeled ticket with no Contract Ledger β appropriate for its type. The two production changes are bug fixes that align behavior with intended contracts (SearchService.ask returning the no-documents answer before requiring a synthesis model) or implement an established hook (buildRequestContext) β not new contract surfaces.
Findings: Pass β no ledger needed, no drift.
πͺ Evidence Audit
- PR body contains a greppable
Evidence:line. - Achieved evidence matches the ACs: L2 unit (16 passed β verified locally below) + L3 integration specs whose achievable ceiling is CI
integration-unified(bucket-D Docker, sandbox-unreachable). - Residuals listed in the PR's
## Post-Merge Validationsection. - Two-ceiling distinction explicit ("local Docker is unavailable in this sandbox").
- No L2βL3 evidence-class inflation in the PR body.
Findings: Pass.
π Cross-Skill Integration Audit
The new integration specs + fixtures/kbTenantRecords.mjs follow the established *.integration.spec.mjs + docker-compose-exec fixture pattern (the PR body explicitly reuses it rather than adding parallel infrastructure). buildRequestContext matches MC's existing override of the documented BaseServer hook. No new convention introduced β no cross-skill reference updates required.
Findings: Pass β no integration gaps.
π§ͺ Test-Execution & Location Audit
- Branch checked out locally:
codex/11645-kb-tenant-isolation-integration@eb5a4ab7f. - Canonical Location: integration specs in
test/playwright/integration/, the fixture in.../integration/fixtures/, the unit spec intest/playwright/unit/ai/services/knowledge-base/β all canonical perunit-test.md. - Ran the changed + related unit tests:
npm run test-unit -- SearchService.noModel.spec.mjs KnowledgeBase.TenantIsolation.spec.mjsβ 16 passed (697ms). The 2 reshapedSearchService.noModelcases (no-docs β empty answer before model; with-refs β still requires model) + the 14 #11632 fail-closed cases all green. - Integration specs: bucket-D Docker β skip locally without Docker; CI
integration-unifiedis the achievable ceiling and is green.
Findings: Tests pass.
π‘οΈ CI / Security Checks Audit
Ran gh pr checks 11703: all 6 checks pass β Analyze (javascript), CodeQL, check, integration-unified (6m4s), lint-pr-body, unit. No pending, no failing.
Findings: Pass β all checks green.
π Required Actions
No required actions β eligible for human merge.
(Two non-blocking items noted above for the author / merge-gate to weigh: the SSE originAgentIdentity provenance gap (Depth Floor) and the test(kb): commit type vs the production fix it carries (Graph Ingestion Notes). Neither blocks merge.)
π Evaluation Metrics
[ARCH_ALIGNMENT]: 90 βbuildRequestContextcorrectly implements the establishedBaseServer/TransportServiceSSE hook to MC parity; integration specs follow the compose-fixture pattern; theSearchService.askreorder aligns with the intended contract. 10 deducted: KB's hook omits theagentIdentityNodeIdMC's sets, leaving the SSEoriginAgentIdentityprovenance gap.[CONTENT_COMPLETENESS]: 88 β thorough Fat-Ticket body (Deltas, Evidence, Post-Merge Validation, head-by-head CI history);buildRequestContext+ fixture functions carry precise JSDoc. 12 deducted: thetest(kb):commit type undersells the production SSE fix the PR carries.[EXECUTION_QUALITY]: 90 β 16 related unit tests verified green locally, CI all-green; surgical changes; integration specs use unique per-run ids +finally-block cleanup. 10 deducted: theoriginAgentIdentitySSE gap.[PRODUCTIVITY]: 92 β achieves #11645's deliverable and fixes 2 real bugs the integration run surfaced. 8 deducted for the residual provenance gap.[IMPACT]: 85 β Phase 5B integration-test parity for the cloud KB epic plus a security-relevant SSE tenant-isolation fix (without it, SSE deployments bypassed the tenant filters entirely).[COMPLEXITY]: 68 β moderate: spans the KB server, KB service, 3 integration specs, a container-exec fixture, and a unit spec; the SSE-transport interaction requires understanding theBaseServerhook +RequestContextServiceflow.[EFFORT_PROFILE]: Heavy Lift β high cross-layer complexity (server + service + Docker fixture + SSE transport) with high impact (epic test parity + a security fix), converged across 3 CI iterations.
Clean Phase 5B work β approved. Nothing blocks merge; the originAgentIdentity SSE provenance gap and the commit type are noted for the author / merge-gate to weigh. Per Β§0 Critical Gate 1, approval = merge-eligibility, not merge authority β human merge-gate (@tobiu).
Resolves #11645
Authored by GPT-5 Codex (Codex Desktop). Session d13c94dd-e721-4e28-ac9e-4d0b3c0f66de.
FAIR-band: over-target [17/30] - taking this lane despite over-target because the operator requested continued two-lane AFK progress, Claude owns #11637, #11645 was unassigned, and #11632 has landed so the KB tenant-isolation integration parity gap is now immediately actionable.
Adds dockerized Knowledge Base tenant-isolation integration coverage on top of the #11632 fail-closed unit suite. The new specs seed deterministic tenant-tagged KB Chroma records in the unified integration fixture, exercise KB MCP tools through proxy identities, and verify private tenant records stay hidden while
neo-sharedcontent remains visible across tenants.Evidence: L2 local static/unit verification + L3 integration specs authored; GitHub
integration-unifiedis the close-target because local Docker is unavailable in this sandbox. The first GitHub L3 run exposed that KB SSE transport was not propagating proxy/OIDC identity intoRequestContextService; this PR now patches KBServer.buildRequestContext()so the existing #11632 tenant filters execute under SSE. Later CI passes exposed two test-shape assumptions:SearchService.noModel.specexpected model-guard fail-fast before retrieval, and the integration hierarchy assertion assumeddocs/output/class-hierarchy.jsonexists in the Docker image. Both assertions are now aligned with the intended contracts.Deltas from ticket
KBTeamPrivateRetrieval.integration.spec.mjsandKBCrossTenantIsolation.integration.spec.mjsas requested.KBAuthRejection.integration.spec.mjsinstead of creating a separateKBAuthRejection.tenant.integration.spec.mjs, keeping the auth rejection surface colocated with its Phase 5A sibling.fixtures/kbTenantRecords.mjsto seed and clean deterministic KB records directly inside the deployedkb-servercontainer, reusing the established compose fixture pattern rather than adding parallel infrastructure.ai/mcp/server/knowledge-base/Server.mjs; without this, authenticated proxy/OIDC requests fell through to the single-tenant path and the integration specs correctly failed.SearchService.ask()so an identity-scoped query with no visible references returns the empty-reference answer before requiring a synthesis model.get_class_hierarchyis validated as tenant-independent by comparing identity-scoped raw tool envelopes, since Docker CI may lack the generated hierarchy JSON but both tenants must still receive identical behavior.Test Evidence
node --check ai/mcp/server/knowledge-base/Server.mjsnode --check test/playwright/integration/fixtures/kbTenantRecords.mjsnode --check test/playwright/integration/KBTeamPrivateRetrieval.integration.spec.mjsnode --check test/playwright/integration/KBCrossTenantIsolation.integration.spec.mjsnode --check test/playwright/integration/KBAuthRejection.integration.spec.mjsnpm run test-unit -- test/playwright/unit/ai/services/knowledge-base/SearchService.noModel.spec.mjs test/playwright/unit/ai/services/knowledge-base/KnowledgeBase.TenantIsolation.spec.mjs- 16 passednpm run test-integration-unified -- test/playwright/integration/KBTeamPrivateRetrieval.integration.spec.mjs test/playwright/integration/KBCrossTenantIsolation.integration.spec.mjs test/playwright/integration/KBAuthRejection.integration.spec.mjs- sandboxed run hitlisten EPERM: operation not permitted 127.0.0.1:13090; escalated rerun started the harness and skipped 4/4 because Docker is unavailable locallyintegration-unifiedrun one79e56684failed 3/34, proving KB SSE identity context was not reaching tenant filters; amended headb0bbade50contains the request-context fix and is awaiting CI.integration-unifiedrun onb0bbade50failed 1/34, provingask_knowledge_basestill requiredGEMINI_API_KEYbefore returning the no-visible-docs answer; amended head46fe59d40contains theSearchService.ask()retrieval-before-model fix and is awaiting CI.46fe59d40failed fulluniton the staleSearchService.noModel.specexpectation and failedintegration-unifiedon the generated hierarchy JSON being absent; amended headeb5a4ab7contains the assertion fixes and is awaiting CI.git diff --cached --checkPost-Merge Validation
integration-unifiedruns the three targeted KB integration specs in Docker and passes without local-sandbox skips.Commit
eb5a4ab7-test(kb): add tenant isolation integration coverage (#11645)Related