LearnNewsExamplesServices
Frontmatter
id12114
titleProvider-agnostic context-limits via localModels.{chat,embedding} role-keyed split + retire SessionService hidden default
stateClosed
labels
enhancementaiarchitecturemodel-experience
assigneesneo-opus-ada
createdAtMay 27, 2026, 11:56 PM
updatedAtJun 21, 2026, 3:43 PM
githubUrlhttps://github.com/neomjs/neo/issues/12114
authorneo-opus-ada
commentsCount0
parentIssue12101
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtMay 28, 2026, 12:35 AM

Provider-agnostic context-limits via localModels.{chat,embedding} role-keyed split + retire SessionService hidden default

Closed v13.0.0/archive-v13-0-0-chunk-14 enhancementaiarchitecturemodel-experience
neo-opus-ada
neo-opus-ada commented on May 27, 2026, 11:56 PM

Premise

6 consumer sites hardcode aiConfig.openAiCompatible.contextLimitTokens / safeProcessingLimitTokens. The actual model-context-window axis is model-role (chat vs embedding), not provider-namespace. For deployments using ollama as graphProvider, friction telemetry reflects openAiCompatible's 262144 regardless of the actual Ollama context window → wrong substrate, wrong calibration.

Operator direction (synthesis): remote providers (gemini) are API-bound — no operator control over context cap, so no template keys needed. Local-model context caps belong on the model-role axis, shared across local providers — one config for chat, one for embedding. Provider-namespace duplication is over-the-top.

Substrate shape (cross-family V-B-A converged with @neo-gpt)

  • Remote providers (gemini, future API-only): no contextLimitTokens in template
  • Local-model-role-keyed: localModels.chat.{context,safe}LimitTokens + localModels.embedding.{context,safe}LimitTokens
  • Namespace boundary (localModels.*) states the actual constraint: locally controlled context caps; top-level chat/embedding would be too close to provider-selector primitive

Surface enumeration (V-B-A 2026-05-27)

Site Role Current state
SemanticGraphExtractor.mjs:125-126 chat pre-existing local vars (cascade fix point)
TopologyInferenceEngine.mjs:83-84 chat added in PR #12113 (consistent with pre-existing pattern)
SessionService.mjs:612-613 chat pre-existing + hidden default fallback || 32768 (violates §no_hidden_default_fallbacks)
mcp/server/memory-core/config.template.mjs:186,197 derivation config-resolved doc-baseline

Template current state: aiConfig.openAiCompatible.contextLimitTokens=262144 + safeProcessingLimitTokens=200000. aiConfig.ollama namespace has NO context-limit keys.

Contract Ledger

Surface Before After Consumer
Chat context limits aiConfig.openAiCompatible.contextLimitTokens=262144 + safeProcessingLimitTokens=200000 aiConfig.localModels.chat.contextLimitTokens=262144 + safeProcessingLimitTokens=200000 (preserve current values per AC1) SemanticGraphExtractor + TopologyInferenceEngine + SessionService chat-path consumers
Embedding context limits NOT present in template aiConfig.localModels.embedding.{context,safe}LimitTokens (concrete value per AC6 V-B-A) Future TextEmbeddingService consumer (pre-positions for #11393/#11395 retry-on-unload telemetry)
Hidden default fallback SessionService.mjs:612 || 32768 Loud-fail on missing config (eliminates §no_hidden_default_fallbacks contract violation)

Acceptance Criteria

  • AC1: localModels.chat.contextLimitTokens=262144 + safeProcessingLimitTokens=200000 added to ai/config.template.mjs (preserves current shipped defaults per @neo-gpt V-B-A)
  • AC2: localModels.embedding.{context,safe}LimitTokens added to ai/config.template.mjs with concrete defaults — value requires V-B-A per AC6
  • AC3: 4 consumer sites migrated from aiConfig.openAiCompatible.{context,safe}* to aiConfig.localModels.{chat,embedding}.* based on consumer role
  • AC4: SessionService.mjs:612 hidden default fallback (|| 32768) deleted; loud-fail on missing config (matches feedback_no_hidden_default_fallbacks operator contract)
  • AC5: SemanticGraphExtractor.mjs:121 JSDoc rationale (openAiCompatible-as-conservative-baseline-for-both-families) rewritten to reflect role-keyed substrate; pre-existing ticket-anchor archaeology (#11965 AC5 / #12059) stripped per AGENTS_ATLAS.md §15.6
  • AC6: Embedding context-cap concrete default value verified against the configured embedding model (LM Studio qwen3-8b-embedding OR Ollama equivalent) BEFORE implementing PR opens. Placeholder 8192/6144 acceptable in template if V-B-A defers to follow-up.
  • AC7: Open question — aiConfig.openAiCompatible.contextLimitTokens legacy retention strategy (keep as deprecated passthrough vs delete in same PR). Operator/Stage-1 sequencing call.

Timing + scope

Recommendation (lean B-with-escape-hatch per @neo-gpt): fold into Epic #12101 Stage 1+ rewrite to avoid double migration. Stage 1 (#12103) establishes meta-leaf primitives; Stage 2 (#12104) rewrites template; this ticket's substrate semantic naturally rides on Stage 2's template rewrite. If operator requires the SessionService hidden-default-fallback retirement (AC4) before Stage 1/2 lands, carve a tiny interim PR with explicit retirement path documented.

Empirical anchor

PR #12113 (just-shipped) preserved the existing aiConfig.openAiCompatible.contextLimitTokens reference for interim consistency with pre-existing pattern. Operator surfaced the substrate-shape issue during cycle-1 V-B-A: "client wants to use ollama". This ticket retires those references as the next substrate-evolution step.

Related

  • Parent: Epic #12101 (greenfield aiConfig substrate redesign — graduated from Discussion #12100)
  • Sibling: #12104 (Stage 2 template rewrite — this ticket's substrate semantic rides on Stage 2)
  • Sibling: #12090 (provider multi-model coexistence; orthogonal but adjacent)
  • Sibling: #11976 (graphProvider routing — the consumer-side discriminator)
  • PR #12113 (interim consistency baseline; this ticket retires the deferred reference)

Discipline anchor

@neo-gpt's V-B-A surfaced: chat default = preserve current 262144/200000 (already live after #12064). Embedding default = NOT memory-asserted — requires V-B-A against actual model before AC6 closure. SessionService.mjs:612 hidden default fallback retirement (AC4) = highest-value invariant per feedback_no_hidden_default_fallbacks operator contract (2026-05-27).

tobiu referenced in commit ea1d481 - "feat(ai): split context-limits into localModels.{chat,embedding} role-keyed config (#12114) (#12115) on May 28, 2026, 12:35 AM
tobiu closed this issue on May 28, 2026, 12:35 AM