LearnNewsExamplesServices
Frontmatter
id10924
titleBucket G: additional unit-test substrate failures discovered post-#10903 Phase 2
stateClosed
labels
bugaitesting
assignees[]
createdAtMay 7, 2026, 8:32 PM
updatedAtMay 12, 2026, 4:10 AM
githubUrlhttps://github.com/neomjs/neo/issues/10924
authorneo-opus-4-7
commentsCount13
parentIssuenull
subIssues
10934 FileSystemIngestor.spec flakes on shared GraphService SQLite-close under workers:1
10935 TransportService.spec:23 still flakes after #10930 bind-race fix — second race surface
10936 G5#2: KBRecorderService.spec singleton-data pollution under workers:1
10937 G5#3: PermissionService.spec AGENT:* singleton cross-pollution under workers:1
10938 G6: Database.spec did-not-run — SQLite.initSchema closed-db reuse under workers:1
10941 G5#5: GraphService.spec:107 residual flake post-TestLifecycleHelper migration under workers:1
10942 PullRequestService.spec:250 flake — getPullRequestDiff file parameter filter under workers:1
10946 DreamServiceGoldenPath.spec:77 residual flake post-#10940 engine=''hybrid'' fix
subIssuesCompleted8
subIssuesTotal8
blockedBy[]
blocking[]
closedAtMay 11, 2026, 11:28 PM

Bucket G: additional unit-test substrate failures discovered post-#10903 Phase 2

Closedbugaitesting
neo-opus-4-7
neo-opus-4-7 commented on May 7, 2026, 8:32 PM

Context

#10903 Phase 1 audit identified ~30 unit-test failures across Buckets A-F. Phase 2 fixes for all 6 buckets landed (#10907, #10910, #10919, #10920, #10921). Lane C followup PR #10922 re-added the unit matrix row to test the cascade closure — exposed a NEW set of unit-test substrate failures NOT covered by Buckets A-F.

Per @neo-gpt's empirical CI inspection on run 25514310691: 7 failed, 4 flaky, 72 skipped, 27 did not run, 1 non-test error despite NEO_TEST_SKIP_CI=true.

The Problem (Bucket G)

Failure classes NOT covered by existing buckets:

G1: Missing required environment variables: GEMINI_API_KEY

3 test failures throwing from Neo.ai.provider.Gemini when GEMINI_API_KEY env var is absent in CI. Tests instantiate the Gemini provider eagerly without skip-guarding.

G2: ChromaConnectionError: Failed to connect to chromadb

3 test failures expecting a live Chroma instance. Tests don't currently skip-guard for missing Chroma daemon.

G3: SqliteError: disk I/O error

3 test failures when SQLite test-DB writes fail under CI sandbox restrictions. Adjacent shape to #10714 (Codex sandbox bootstrap probe).

G4: Namespace collision in unitTestMode for Neo.ai.mcp.server.github-workflow.Config

1 test failure — Neo class registration collision under unitTestMode. Could be test-isolation issue (parallel-test contamination) OR real bug in class-loading.

G5: 4 flaky tests (intermittent failures)

Flaky — not deterministic. May be timing-sensitive substrate or genuine race conditions.

G6: 27 "did not run"

Tests skipped at the runner level due to upstream failure. Phase 1 diagnostic needed: are these all blocked by setup of one common substrate, or independent?

The Fix (Investigation-Shaped)

This is investigation-shaped — root causes vary across G1-G6.

Phase 1 (this ticket): per-class triage

For each of G1-G5, determine:

  • Whether to skip-guard via NEO_TEST_SKIP_CI (canonical bucket-skip pattern)
  • Whether to fix at substrate (e.g., add config defaults that make the test substrate self-bootstrapping)
  • Whether to test-fixture migrate (e.g., mock the dependency)

For G6, identify the upstream-blocking root and address it.

Phase 2 (sub-tickets): per-class fixes

Each G1-G6 likely warrants its own small PR.

Phase 3 (final): re-attempt Lane C unit matrix row addition

Once Phase 2 complete + #10923 setMcpServer regression fixed, re-file Lane C followup PR adding unit row.

Acceptance Criteria

  • Phase 1 triage table produced (each failing test → bucket G1/G2/G3/G4/G5/G6 + fix path).
  • Phase 2 sub-tickets filed per class.
  • Phase 3 Lane C followup PR re-files; unit matrix row passes in CI.

Out of Scope

  • #10923 setMcpServer regression (filed concurrently — different shape, code regression not test-substrate gap).
  • Integration-spec deferrals from Bucket F (already handled via #10919 + #10920).

Related

  • Predecessor: #10903 (Buckets A-F audit).
  • Sibling: #10923 (setMcpServer regression).
  • Surfacing context: Lane C followup PR #10922 closure attempt (CI exposed Bucket G).
  • Adjacent shape: #10714 (Codex sandbox SQLite — possibly G3 root).

Origin Session ID: 7e897a0b-33ce-4d6c-b1a9-a1ff93e4e571

Retrieval Hint: query_raw_memories(query="Bucket G unit-test failures GEMINI_API_KEY ChromaConnection SqliteError namespace collision Phase 1 triage")

tobiu referenced in commit e66b8af - "fix(test): correct bucket G1-G3 skip-guards for 5 hard failures (#10924) (#10928) on May 7, 2026, 10:49 PM
tobiu referenced in commit 576652b - "test(github-workflow): isolate config template namespace (#10924) (#10929) on May 7, 2026, 11:22 PM
tobiu referenced in commit 98897fc - "feat(ci): re-add unit suite to matrix post-Bucket-G substrate (#10939) (#10953) on May 8, 2026, 2:43 PM