LearnNewsExamplesServices
Frontmatter
id9929
titleStabilize AI Test Suite environment dependencies
stateClosed
labels
enhancementai
assigneestobiu
createdAtApr 12, 2026, 6:23 PM
updatedAtApr 12, 2026, 6:30 PM
githubUrlhttps://github.com/neomjs/neo/issues/9929
authortobiu
commentsCount0
parentIssue9922
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtApr 12, 2026, 6:30 PM

Stabilize AI Test Suite environment dependencies

tobiu
tobiu commented on Apr 12, 2026, 6:23 PM

Architectural Context

During the completion of Epic #9922, testing the Two-Pillar RAG architecture locally revealed intermittent failures strictly related to local LLM daemon execution and parallel test execution:

  1. Authorization.spec.mjs failed due to port conflicts (EADDRINUSE: 3333) acting against a globally booted ai:server.
  2. Librarian.spec.mjs failed occasionally on rate limits with Google Gemini when hammering ask_knowledge_base concurrently.
  3. QA.spec.mjs crashed strictly when openAiCompatible inferences failed locally (e.g. absent LLM).

Corrective Action

  • Mapped MCP_SSE_PORT to 5555 in Authorization.spec.mjs.
  • Wrapped graphRAG processEvent in Librarian.spec.mjs to dynamically fallback to test.skip if the inference times out, preventing hard suite crashes.
  • Adjusted assertions in QA.spec.mjs to natively identify absent LLMs and skip expectations gracefully to prevent 100% loss test suite states if Ollama or MLX is not running.
tobiu added the enhancement label on Apr 12, 2026, 6:23 PM
tobiu added the ai label on Apr 12, 2026, 6:23 PM
tobiu added parent issue #9922 on Apr 12, 2026, 6:23 PM
tobiu referenced in commit 20a29c8 - "test: Stabilize AI Test Suite environment dependencies (#9929)" on Apr 12, 2026, 6:23 PM
tobiu cross-referenced by PR #9930 on Apr 12, 2026, 6:24 PM
tobiu assigned to @tobiu on Apr 12, 2026, 6:24 PM
tobiu referenced in commit f6967de - "test: Stabilize AI Test Suite environment dependencies (#9929) (#9930)" on Apr 12, 2026, 6:30 PM
tobiu closed this issue on Apr 12, 2026, 6:30 PM