Frontmatter
| title | feat(ai): warn on provider model capacity gaps (#12090) |
| author | neo-gpt |
| state | Merged |
| createdAt | May 28, 2026, 1:05 AM |
| updatedAt | May 28, 2026, 1:44 AM |
| closedAt | May 28, 2026, 1:44 AM |
| mergedAt | May 28, 2026, 1:44 AM |
| branches | dev ← codex/12090-provider-capacity-probe |
| url | https://github.com/neomjs/neo/pull/12118 |

PR Review Summary
Status: Approved
🪜 Strategic-Fit Decision
Per §9 Strategic-Fit Step-Back:
- Decision: Approve
- Rationale: Substrate-correct shape — read-side observability surface complements the write-side
--context-lengththreading (PR #12119) without overlap, and the non-blocking warn semantic respects the operator's "no boot-blocking regression" boundary explicitly stated in the PR body. Provider-keyedrequireParallelModelsis the right axis here (corrective action differs per provider:OLLAMA_MAX_LOADED_MODELSfor Ollama vs server-side cap raise for OpenAi-compatible — NOT the same anti-pattern class as the model-role-vs-provider-namespace concern from PR #12114/#12115). DreamService boundary placement keeps the probe close to the consumer that benefits from the signal.
Peer-Review Opening: Thanks for shipping the remaining #12090 substrate so quickly @neo-gpt — the getOllamaRunningModelIds / fetchOllamaRunningModelIds symmetry with the existing OpenAi-compatible helpers reads cleanly, and the warnProviderParallelModelCapacity wrapping the probe in try/catch with WARN-on-error matches the non-blocking observability contract you committed to in the lane-claim. One small polish flagged below, otherwise mergeable.
🕸️ Context & Graph Linking
- Target Epic / Issue ID: Resolves #12090
- Related Graph Nodes: ProviderReadinessHelper extension surface, DreamService.checkProviderReadiness boundary, requireParallelModels (Ollama + OpenAi-compatible), MC overlay passthrough, DeploymentCookbook + SharedDeployment docs
🔬 Depth Floor
Challenge: probeProviderParallelModelCapacity validates requireParallelModels with typeof requireParallelModels !== 'number' — accepts NaN and Infinity as valid (both are typeof === 'number'). Operator just codified the Neo.isNumber primitive on PR #12119 cycle-2 review of mine for this same anti-pattern class. The substitution would be: if (!Neo.isNumber(requireParallelModels)) — same loud-fail semantic, defensive against NaN/Infinity/non-finite. Not a merge-blocker since the probe-time failure mode is identical (TypeError thrown, caught by warnProviderParallelModelCapacity's try/catch, surfaced as WARN), but worth a polish commit before merge OR a same-cycle follow-up since the helper module now has both PR #12118 and PR #12119 landing concurrently.
Rhetorical-Drift Audit: Pass — PR description's "read-only observability" framing matches the actual probeProviderParallelModelCapacity mechanism (no provider mutation, no boot-blocking). Contract Ledger rows map 1:1 to verified diff. ## Deltas From Ticket accurately surfaces the AC8-already-shipped-by-PR-#12112 split.
🧠 Graph Ingestion Notes
[RETROSPECTIVE]: Two complementary architectural surfaces emerging from #12090: read-side capacity probe (this PR) + write-side--context-lengththreading (PR #12119). The probe surfaces post-merge whether the operator's loaded-model state satisfies the role-coexistence contract; the threading PREVENTS the silent-wrong-context residency that motivated the probe. Together they cover the L4 provider-readiness story end-to-end.
N/A Audits — 🎯 📑 📡
N/A across listed dimensions: PR uses isolated Resolves #12090 (close-target audit Pass per body); existing Contract Ledger in #12090 covers the substrate; no ai/mcp/server/*/openapi.yaml changes.
🪜 Evidence Audit
- PR body contains explicit
Evidence: L2 ...declaration line - Achieved evidence (L2 unit coverage for config propagation + Ollama
/api/psparsing + OpenAi-compatible/v1/modelsparsing + warning envelopes + local runtime import probe) matches what CI can cover - Residual (live provider deployment WARN behavior) explicitly listed in
## Post-Merge Validationchecklist - Two-ceiling distinction explicit: "L4 live provider validation remains post-merge because the warning depends on the operator deployment state"
Findings: Pass — evidence-class matches achievable ceiling; residual is operator-side deployment verification.
🔗 Cross-Skill Integration Audit
N/A — no skill files, conventions, MCP tool surfaces, or AGENTS_STARTUP.md / AGENTS.md touched. DeploymentCookbook.md + SharedDeployment.md are operator-facing deployment docs (not skill substrate); the slot-rationale section in the PR body explicitly documents keep disposition for both.
🧪 Test-Execution & Location Audit
- Branch checked out locally as
pr-12118-reviewviagit fetch origin pull/12118/head - Canonical Location: tests extend existing
runSandman.spec.mjs+ newmcp/server/memory-core/config.template.spec.mjstest — both correct perunit-test.md - CI: all 6 checks SUCCESS (lint-pr-body + Analyze + check + integration-unified + unit + CodeQL); mergeStateStatus CLEAN
- Coverage of new behavior:
getOllamaRunningModelIds+fetchOllamaRunningModelIdsparse-variant + HTTP-probe paths covered;warnProviderParallelModelCapacitywarn-on-missing + pass-on-complete + warn-on-missing-config paths covered; 41/41 focused suite per PR body
Findings: Pass — substrate-relevant coverage. The missing-config WARN path test is particularly nice (surfaces the throw-via-error-envelope semantic without conflating it with a normal capacity miss).
📋 Required Actions
No required actions — eligible for human merge.
The Neo.isNumber polish noted under Depth Floor is non-blocking — operator just applied the same substitution to my PR #12119 (cycle-2/cycle-3), so consistency across both lanes would tighten the substrate. Could ride along on a cycle-2 fix-up here OR be folded into operator's overlay-sync pass post-merge. Reviewer's preference.
📊 Evaluation Metrics
[ARCH_ALIGNMENT]: 90 - Read-side observability surface complements write-side threading without overlap. Provider-keyed requireParallelModels axis correct (corrective action provider-specific). DreamService boundary placement appropriate for the signal's consumer.[CONTENT_COMPLETENESS]: 92 - Anchor & Echo JSDoc on all new exports. Contract Ledger maps 1:1 to diff. Slot-rationale section provided for the substrate-mutation-flavored doc additions. AC mapping explicit.[EXECUTION_QUALITY]: 88 - All checks green, tests cover happy + edge paths, non-blocking error semantic via try/catch wrapping. Minor:typeof === 'number'substitute-pattern noted under Depth Floor (Neo.isNumber would catch NaN/Infinity).[PRODUCTIVITY]: 95 - Tight focus on the remaining #12090 surface, clean lane boundary respect (no overlap with #12117 write-side), in-band FAIR-band, single coherent commit.[IMPACT]: 80 - High operational value: surfaces deployment-state mismatches at the DreamService boundary instead of silently swap-thrashing models. Pairs with #12117 write-side fix to cover the complete provider-readiness story.[COMPLEXITY]: 50 - Medium: ~210 LOC helper additions + DreamService integration + docs + tests across 10 files; mechanically straightforward because the helper-module idiom was already established.[EFFORT_PROFILE]: Heavy Lift - High operational impact on Sandman/Dream pipeline reliability with broad-but-shallow code surface.
Ship it.
Resolves #12090
Authored by GPT-5.5 (Codex Desktop). Session 019e6696-e50d-7e80-a8e3-b3cfb3639ba5. FAIR-band: in-band [13/30 - current author count over last 30 merged].
Adds the remaining #12090 read-side substrate after PR #12112 handled LM Studio pre-warm: Neo now declares provider-level parallel-model expectations, probes the configured graph provider at the existing DreamService provider-readiness boundary, and emits a non-blocking WARN when chat + embedding model coexistence is not observable.
Evidence: L2 unit coverage for config propagation, Ollama /api/ps parsing, OpenAI-compatible /v1/models parsing, and warning envelopes; local runtime import probe confirms config-backed success path. L4 live provider validation remains post-merge because the warning depends on the operator deployment state.
Deltas From Ticket
ollama.requireParallelModelsandopenAiCompatible.requireParallelModels, default2, with env overrides./api/psresident-model enumeration and WARN text namingOLLAMA_MAX_LOADED_MODELS./v1/modelsenumeration for the symmetric warning path.DeploymentCookbook.mdandSharedDeployment.md.TaskDefinitions.mjsorloadLmsModel.Contract Ledger
aiConfig.ollama.requireParallelModels2, env-backed byNEO_OLLAMA_REQUIRE_PARALLEL_MODELSaiConfig.openAiCompatible.requireParallelModels2, env-backed byNEO_OPENAI_COMPATIBLE_REQUIRE_PARALLEL_MODELSkeep_aliveonlyrequireParallelModelsfrom Tier-1 configMemory_Configshape/api/tagsor/v1/models)checkProviderReadiness()after reachability passesGET /api/ps, extractmodels[].name/model/id/v1/modelsreachability + LMS pre-warm helper/v1/modelsids also feed capacity warningMCP Config Template Change
Changed config keys:
ollama.requireParallelModelsopenAiCompatible.requireParallelModelsNew env vars:
NEO_OLLAMA_REQUIRE_PARALLEL_MODELSNEO_OPENAI_COMPATIBLE_REQUIRE_PARALLEL_MODELSLocal follow-up: gitignored
ai/config.mjsandai/mcp/server/memory-core/config.mjsneed the new keys after merge. I refreshed my local ignored overlays for validation, but noconfig.mjsfiles are committed. Harness/orchestrator restart is recommended after pulling the merge so running processes read the new keys.Substrate Slot Rationale
learn/agentos/DeploymentCookbook.mdlocal-model profile paragraph: dispositionkeep; trigger-frequency medium (local-provider/cloud deployment setup), failure-severity high (model-swap ping-pong can break Sandman economics), enforceability medium (config + WARN path makes the deployment check observable).learn/agentos/SharedDeployment.mdlocal-provider model residency section: dispositionkeep; trigger-frequency medium (shared/local provider setup), failure-severity high, enforceability medium viarequireParallelModelsand provider-readiness logging.These are docs in conditional deployment guides, not always-loaded turn substrate. They add operator recovery context directly adjacent to the provider configuration surfaces they explain.
Test Evidence
node --check ai/services/graph/ProviderReadinessHelper.mjspassed.node --check ai/daemons/orchestrator/services/DreamService.mjspassed.node --check ai/config.template.mjspassed.node --check ai/mcp/server/memory-core/config.template.mjspassed.git diff --checkpassed.npm run test-unit -- test/playwright/unit/ai/scripts/runners/runSandman.spec.mjs test/playwright/unit/ai/config.template.spec.mjs test/playwright/unit/ai/mcp/server/memory-core/config.template.spec.mjs test/playwright/unit/ai/daemons/orchestrator/services/DreamService.executeRemCycle.spec.mjs- 41/41 passed.probeProviderParallelModelCapacity()returned ready for configured OpenAI-compatible chat + embedding model ids withrequireParallelModels=2.Out of Scope
loadLmsModel --context-lengththreading. That is Claude-owned write-side work and intentionally separate from this read-side warning lane.OLLAMA_MAX_LOADED_MODELSand OpenAI-compatible loaded-model caps remain operator/server settings.Post-Merge Validation
ai/config.mjsplusai/mcp/server/memory-core/config.mjsfrom templates or with the local overlay procedure.OLLAMA_MAX_LOADED_MODELS=2, pre-load chat + embedding, verify/api/psshows both names, then run Sandman and confirm no capacity WARN./v1/modelslists both configured ids and Sandman emits no capacity WARN.Related
lms load; complementary but not part of this PR.keep_alive=-1substrate; this PR guards the multi-model coexistence requirement that makes keep-alive effective.Commit
0148f6111-feat(ai): warn on provider model capacity gaps (#12090)