LearnNewsExamplesServices
Frontmatter
id12763
titleDynamic-import only the configured provider in buildChatModel
stateClosed
labels
enhancementairefactoringperformance
assigneesneo-gpt
createdAtJun 8, 2026, 2:04 PM
updatedAtJun 8, 2026, 4:35 PM
githubUrlhttps://github.com/neomjs/neo/issues/12763
authorneo-opus-vega
commentsCount0
parentIssue12740
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 8, 2026, 4:35 PM

Dynamic-import only the configured provider in buildChatModel

Closed v13.0.0/archive-v13-0-0-chunk-17 enhancementairefactoringperformance
neo-opus-vega
neo-opus-vega commented on Jun 8, 2026, 2:04 PM

Context

ai/provider/buildChatModel.mjs (created in #12741) statically imports all three providers at module-load time:

import { GoogleGenerativeAI } from '@google/generative-ai';
import OllamaProvider           from './Ollama.mjs';
import OpenAiCompatibleProvider from './OpenAiCompatible.mjs';

…but the selector resolves and uses exactly one per the configured modelProvider. @tobiu flagged the load cost as a follow-up to #12741. It is local-first aligned (#12740): a local (ollama / openAiCompatible) deployment currently loads the remote @google/generative-ai SDK it never calls.

The Problem

All three provider modules — including the @google/generative-ai npm package, the heaviest — load whenever buildChatModel.mjs is imported, regardless of which provider the deployment configured. For a local-first deployment, pulling in the remote Gemini SDK is pure waste: load time, memory, and dependency/attack surface. Both the Knowledge Base and Memory Core MCP servers import buildChatModel, so both pay it.

The Architectural Reality

  • ai/provider/buildChatModel.mjs L1-3 — the static imports.
  • The selector has three branches (openAiCompatible / ollama / gemini), each using one provider via injectable factories (ollamaProviderFactory = Neo.create(OllamaProvider, cfg), openAiCompatibleProviderFactory = Neo.create(OpenAiCompatibleProvider, cfg), geminiClientFactory = new GoogleGenerativeAI(...)). The default factories close over the statically-imported modules.
  • Call-site shape matters: SearchService.construct() and Memory Core's SessionService build the model synchronously (buildChatModel returns sync). A naive await import() inside buildChatModel would make it async and ripple to those sync construct() call-sites.

The Fix

Dynamic-import the configured provider so only it loads. Two shapes to evaluate:

  1. Lazy-in-shim (preferred if viable): the openAiCompatible / ollama branches return a {generateContent} shim whose method is already async — await import() the provider there (memoized on first call), keeping buildChatModel synchronous and avoiding any call-site ripple. The gemini branch returns the GoogleGenerativeAI model directly (sync), so it would need its factory to become async (or that branch alone to go async).
  2. Async buildChatModel: make the whole function async (await import() the needed module in-branch) and move the model build to initAsync at the call-sites. Cleaner provider isolation, but a call-site refactor across SearchService / SessionService.

Keep the injectable factories as test seams (specs pass mocked factories — no real import).

Contract Ledger

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
buildChatModel({...}) return ai/provider/buildChatModel.mjs Load only the configured provider's module (dynamic import) Injectable factories stay sync test seams; behavior/routing unchanged The function JSDoc L1-3 static imports + the 3 branches; consumers SearchService.construct, SessionService, MemoryService import it

Decision Record impact

none — load/perf optimization; does not touch the AiConfig SSOT (ADR 0019) or provider routing.

Acceptance Criteria

  • A deployment with modelProvider = ollama or openAiCompatible does not load @google/generative-ai.
  • buildChatModel loads only the configured provider's module.
  • The injectable factory test seams still work (mocked factories, no real import).
  • SearchService / SessionService / MemoryService call-sites stay coherent (sync stays sync, or move to initAsync if the async shape is chosen).
  • Existing buildChatModel + SearchService + HealthService specs still pass.

Out of Scope

  • Changing provider behavior or routing — this is purely when/whether modules load.
  • The embedding-provider path (TextEmbeddingService) — separate surface.

Avoided Traps

  • Do not make buildChatModel async without handling the sync construct() call-sites — that would break SearchService / SessionService construction.
  • Do not drop the injectable factories — they are the test seams.

Related

  • #12741 (created buildChatModel)
  • #12740 (cost-safety epic — local-first alignment)

Origin Session ID: 728442d6-a2a0-4481-a631-a3112ce6d703

Retrieval Hint: "buildChatModel dynamic-import provider lazy load"

Live latest-open sweep: checked latest 30 open issues 2026-06-08; no equivalent found (closest: #12740 cost-safety epic + subs #12741–#12748, none covering provider-import load cost).

tobiu referenced in commit 44599e6 - "fix(ai): lazy-load configured chat provider (#12763) (#12764)" on Jun 8, 2026, 4:35 PM
tobiu closed this issue on Jun 8, 2026, 4:35 PM