Tier local-model chat context default to host RAM (256K needs ~90GB)

Context

The operator (2026-06-15, v13.1 scoping) flagged that the local chat model is configured with a ~200K-token content window that needs ~90GB RAM and will not load on his machine. v13.1 names local-model resource-safety as a reliability cornerstone. This is distinct from #12740 (Gemini provider cost-safety, substantially shipped per its 2026-06-13 triage): that epic addressed which provider runs; this ticket addresses the local model's context-window resource footprint.

The Problem

ai/config.template.mjs:230-231 defaults the local chat model's contextLimitTokens to 262144 (256K) and safeProcessingLimitTokens to 200000 (200K), tuned for gemma-4-31b-it's native 256K context (config comment :213). A 256K-token KV cache for a 31B model needs ~90GB RAM; on a sub-90GB host the model fails to load (or the inference server OOMs), so local-first inference is unavailable by default on common developer hardware — undermining the local-first goal #12740 established.

The values are env-overridable (NEO_LOCAL_MODELS_CHAT_CONTEXT_LIMIT_TOKENS / …_SAFE_PROCESSING_LIMIT_TOKENS), but the default assumes a high-RAM host; a resource-constrained operator must already know the override exists and the right value for their RAM — a discovery-and-tuning burden the default should absorb.

The Architectural Reality

• ai/config.template.mjs:229-232 — Tier-1 AiConfig leaves (localModels.chat.contextLimitTokens / safeProcessingLimitTokens), env-overridable, consumed by ConsumerFrictionHelper.invokeWithGuardrail for context-overflow pre-checks (config comment :215-218).
• The actual model-load context length (the KV cache that drives RAM) is set on the LM-Studio preload path — ai/services/graph/providerReadinessHelper.mjs (loadLmsModel() / buildLmsPreloadConfig(), prior-session evidence) — which should be sized in lock-step with the guardrail leaves so the guardrail never advertises more context than the model was loaded with.
• ADR 0019 (AiConfig reactive Provider SSOT): the fix must read/derive resolved leaves at the use site, not introduce a parallel sizing alias.

The Fix

Make the local-model context default host-aware instead of a fixed 256K:

1. Derive the default contextLimitTokens from detected system memory (os.totalmem()) via a small resolver, with a conservative low-RAM floor (RAM-band tiers, e.g. 32K / 64K / 128K / 256K) — so the model loads out-of-the-box on common hosts, while high-RAM hosts still get the full native context.
2. Keep safeProcessingLimitTokens derived as the ~76% headroom band of the resolved limit (preserve the "explicit value avoids 0.75 × cap derivation drift" intent in the config comment).
3. Size the LM-Studio preload context-length from the same resolved leaf so the loaded KV cache matches the guardrail.
4. Explicit NEO_LOCAL_MODELS_CHAT_* env overrides remain authoritative (operator pin wins over auto-tier).

(Exact RAM→context tiers + whether the resolver is a new helper vs inline are a PR implementation detail; if a new .mjs is introduced, run structural-pre-flight.)

Contract Ledger

Surface-Anchor V-B-A: config.template.mjs:230-231 (262144 / 200000) confirmed by read; providerReadinessHelper.mjs LMS-preload path cited from prior-session evidence (re-confirm at implementation).

Decision Record impact

aligned-with ADR 0019 — derive/read resolved AiConfig leaves at the use site; no parallel provider/sizing alias.

Acceptance Criteria

• Local chat model's default context limit is derived from detected host RAM (not a fixed 256K), with a conservative low-RAM floor that loads on common (<90GB) hardware.
• safeProcessingLimitTokens default tracks the resolved limit's ~76% headroom band.
• LM-Studio preload context-length is sized from the same resolved leaf (loaded KV cache ≤ guardrail).
• Explicit NEO_LOCAL_MODELS_CHAT_* env overrides remain authoritative.
• L2 unit coverage for the RAM→context resolver (bands + floor + env-override precedence).
• (Post-merge, host-only — not CI-verifiable) Operator confirms the bundled local chat model loads on the sub-90GB machine that previously failed.

Out of Scope

• Removing 256K native-context support for high-RAM hosts (they keep it).
• Embedding-model context limits (:251-252, separate 32K/28K band) — unless the resolver cleanly generalizes.
• Provider routing / Gemini cost-safety (#12740 owns that).
• Per-model capacity auto-detection from the model card (future; this ticket sizes by HOST RAM).

Avoided Traps

• Blanket-lowering the 256K default — rejected: penalizes the high-RAM hosts it was tuned for. Tier by host instead.
• Documentation-only (tell operators to set the env) — rejected: the default should absorb the tuning burden; local-first means it loads out-of-the-box.
• A parallel sizing SSOT — rejected per ADR 0019; derive from the resolved leaf.

Related

• #12740 — Agent OS local-first provider defaults + cost-safety (provider routing; substantially shipped; this is the distinct context-window-RAM sizing concern).
• #12456 — AiConfig SSOT cleanup parent.
• ADR 0019 — AiConfig reactive Provider SSOT.

Live latest-open sweep: checked latest 20 open issues at 2026-06-15T21:20Z; no equivalent found. A2A claim sweep (last 30 messages): no competing local-model-context claim.

Release classification: v13.1 reliability cornerstone (operator-flagged friction-1); boardless — milestone/board attachment is the ROADMAP/milestone-#8 owner's call.

Origin Session ID: 47b6dbc0-7673-4ad3-a9f5-bef3b606c56b Retrieval Hint: "local model chat context default 256K 200K contextLimitTokens host RAM tier LMS preload won't load sub-90GB"