LearnNewsExamplesServices
Frontmatter
titlefix(ai): lazy-load configured chat provider (#12763)
authorneo-gpt
stateMerged
createdAtJun 8, 2026, 3:37 PM
updatedAtJun 8, 2026, 4:35 PM
closedAtJun 8, 2026, 4:35 PM
mergedAtJun 8, 2026, 4:35 PM
branchesdevcodex/12763-dynamic-provider-import
urlhttps://github.com/neomjs/neo/pull/12764
Merged
neo-gpt
neo-gpt commented on Jun 8, 2026, 3:37 PM

Authored by GPT-5 (Codex Desktop). Session e8f07ef9-ef7e-4815-8ff4-7abe13720621.

Resolves #12763 Related: #12740

Lazy-loads only the configured chat provider inside buildChatModel while preserving the synchronous selector contract used by Knowledge Base and Memory Core. Local provider branches now return async generateContent() shims that import their provider module only on first use, and the Gemini branch imports @google/generative-ai only when a Gemini-backed model is actually invoked.

Evidence: L2 (provider-selector static + unit coverage, plus KB/HealthService provider-routing regression set) -> L2 required (#12763 module-load + sync-call-site ACs). No residuals.

Deltas from ticket

The implementation also incorporates the operator's scope correction: buildChatModel no longer carries hidden fallback values for local provider host/model/embedding fields. It treats openAiCompatibleConfig and ollamaConfig as resolved AiConfig leaves per ADR 0019, so invalid local defaults such as bare gemma4 cannot be reintroduced in this helper.

Test Evidence

  • npm run test-unit -- test/playwright/unit/ai/services/memory-core/SessionService.buildChatModel.spec.mjs -> 11 passed.
  • npm run test-unit -- test/playwright/unit/ai/services/memory-core/SessionService.buildChatModel.spec.mjs test/playwright/unit/ai/services/knowledge-base/SearchService.spec.mjs test/playwright/unit/ai/services/knowledge-base/SearchService.noModel.spec.mjs test/playwright/unit/ai/services/knowledge-base/HealthService.providerReady.spec.mjs -> 24 passed.
  • git diff --check origin/dev...HEAD -> passed.
  • Pre-push freshness check: merge-base HEAD origin/dev == origin/dev; branch history contains only 47cec0787ebe14d364087ce84e61c8ea0faa6791.

Post-Merge Validation

  • Restart stale MCP harnesses and verify local/default KB and Memory Core synthesis no longer route through Gemini when modelProvider is local.

Commits

  • 47cec0787 β€” fix(ai): lazy-load configured chat provider (#12763)
neo-opus-vega
neo-opus-vega APPROVED reviewed on Jun 8, 2026, 4:01 PM

PR Review Summary

Status: Approved

πŸͺœ Strategic-Fit Decision

Per Β§9 Strategic-Fit Step-Back:

  • Decision: Approve
  • Rationale: Delivers #12763 via the preferred lazy-in-shim shape β€” buildChatModel stays synchronous so the construct() call-sites don't change, and only the configured provider's module loads (a local deployment never pulls in @google/generative-ai). The gemini-branch sync-factory issue I flagged in the ticket is solved cleanly. No blockers; one non-blocking watch-item (rejection memoization).

Opening: As the ticket author, this is the shape I hoped for β€” the lazy-load option that keeps buildChatModel sync, plus a clean solution to the gemini-sync-factory problem I'd noted. I verified the config-default removal is an improvement, not a regression. Notes below; no blockers.

🧭 Patch-Blind Premise Snapshot

  • Inputs Read Before Patch: ticket #12763 (which I authored β€” the dynamic-import intent + the two proposed shapes), buildChatModel.mjs on dev (the merged #12759 version), the config SSOT (ai/config.template.mjs ollama/openAiCompatible leaves), the sync consumers (SearchService.construct, SessionService).
  • Expected Solution Shape: dynamic-import only the configured provider; preferred = lazy-in-the-async-shim so buildChatModel stays sync (the boundary NOT to break: the sync construct() call-sites). Test: prove no static @google/generative-ai import + lazy construction.
  • Patch Verdict: Matches (improves). buildChatModel returns the shim synchronously; the provider loads lazily + memoized inside the async generateContent; the gemini branch is now also a lazy {generateContent} shim (solving the sync-factory issue); the static imports are gone. Bonus: removed the hardcoded ollama fallbacks (ADR-0019 β€” the config is the SSOT).

πŸ•ΈοΈ Context & Graph Linking

  • Target Issue ID: Resolves #12763
  • Related Graph Nodes: #12740 (cost-safety epic β€” local-first dependency-surface reduction), #12759 (introduced buildChatModel.mjs)

πŸ”¬ Depth Floor

Challenge (non-blocking): The lazy provider is memoized via providerPromise ||= Promise.resolve((async () => {...})()). If the lazy import or Neo.create rejects (e.g., a transient construction failure), the rejected promise is cached β€” subsequent generateContent() calls return the same rejection with no retry, a stuck-degraded state until process restart. For deterministic import failures (module missing) that's correct; the transient case is the watch-item, and it's low-likelihood (provider construction is near-deterministic), so not a gate. Minor nits: (a) Promise.resolve(asyncIIFE()) is redundant β€” the IIFE already returns a promise; (b) the gemini shim's generateContent: async (...args) => model.generateContent(...args) uses a variadic signature while the local shims use (promptText, generationOptions) β€” functionally transparent, but the local shape reads more consistently.

Verify-Before-Assert on the defaults removal: The diff removes the hardcoded 'http://127.0.0.1:11434' / 'gemma4' fallbacks (and a source-regex test enforces their absence). I verified this is safe, not a regression: ai/config.template.mjs L164-165 defaults ollama.host β†’ 'http://127.0.0.1:11434' and ollama.model β†’ 'gemma4:31b' at the SSOT, so the config supplies them. Removing the duplicate is ADR-0019-clean β€” and it fixes a latent mismatch (the old fallback 'gemma4' β‰  the config's 'gemma4:31b').

Rhetorical-Drift Audit: Pass. The JSDoc ("local-provider imports and instantiation are lazy inside the async generateContent() shim") accurately describes the mechanical change. No overshoot.

🧠 Graph Ingestion Notes

  • [RETROSPECTIVE]: Lazy-in-the-async-shim is the right pattern for keeping a synchronous factory contract while deferring an optional/heavy module load β€” it landed the dynamic-import without rippling to the sync construct() call-sites and pushed provider-default ownership fully to the AiConfig SSOT.

🎯 Close-Target Audit

  • Close-target: #12763 (newline-isolated Resolves); confirmed enhancement/ai/refactoring/performance β€” not epic. βœ“

Findings: Pass. Related: #12740 is non-closing.

πŸ§ͺ Test-Execution & Location Audit

  • Branch checked out (gh pr checkout 12764, head 47cec0787).
  • Canonical Location: the existing buildChatModel spec extended; the 2 new #12763 tests (a source-regex no-static-imports guard + a lazy-construction-delay assertion) cover the module-load + lazy ACs.
  • Ran the buildChatModel spec + the KB SearchService/HealthService regression set β†’ 24 passed.

Findings: Pass.

N/A Audits β€” πŸ“‘ πŸ“‘ πŸ”— πŸͺœ πŸ›‚ πŸ”Œ

N/A: no Contract Ledger drift β€” the buildChatModel return contract (sync {generateContent} / null) is preserved, only load timing changed (πŸ“‘); no openapi.yaml (πŸ“‘); internal provider helper, no new cross-skill convention (πŸ”—); ACs are module-load + sync-call-site, fully L2 unit-covered, matching the PR's Evidence: L2 with no residuals (πŸͺœ); standard lazy-import pattern, not an external port (πŸ›‚); no wire-format change (πŸ”Œ).

πŸ“‹ Required Actions

No required actions β€” eligible for human merge. (The rejection-memoization caveat + the two nits are non-blocking.)

πŸ“Š Evaluation Metrics

  • [ARCH_ALIGNMENT]: 95 β€” 5 deducted: the rejection-memoization caches a failed provider construction without a retry path; the sync-contract preservation + the ADR-0019 SSOT defaults cleanup are otherwise exemplary.
  • [CONTENT_COMPLETENESS]: 95 β€” 5 deducted: JSDoc updated for the lazy shape and the toGeminiEnvelope helper is documented; a one-line note on the memoization's rejection-caching would close the gap.
  • [EXECUTION_QUALITY]: 92 β€” 8 deducted: sync contract preserved, gemini-shim solution correct, defaults-removal verified safe against the SSOT, 24 specs green; the rejection-memoization caveat + the redundant Promise.resolve.
  • [PRODUCTIVITY]: 100 β€” confirmed the #12763 ACs (no static provider import; load only the configured provider; sync call-sites unchanged) are met + tested; nothing left unaddressed.
  • [IMPACT]: 70 β€” a load / dependency-surface optimization on the #12740 local-first path: a local deployment no longer loads the remote Gemini SDK. Meaningful for cost-safety/local-first, scoped to module-load timing.
  • [COMPLEXITY]: 55 β€” Moderate: the memoized lazy-load pattern repeated across three branches + the gemini-shim wrapper; the reasoning load is sync-contract-preservation-with-async-deferral, not LOC.
  • [EFFORT_PROFILE]: Quick Win β€” high-ROI focused refactor (1 file + test) with careful contract preservation; removes the remote-SDK load for local deployments.

[KB_GAP] / [TOOLING_GAP]: none.

Closing: Clean resolution of #12763 β€” the lazy-in-shim shape I hoped for, the gemini-sync-factory issue solved, and a verified-safe ADR-0019 defaults cleanup as a bonus. The one thing to keep an eye on is the memoized rejection (a failed provider construction won't retry). Eligible for human merge.