Raise OpenAiCompatible contextLimitTokens to match gemma-4 native 256K

Substantive-Rationale Exception — Ungraduated-Discussion Cross-Check

Per ticket-create-workflow §1c: this ticket cites Discussion #12062 which has NOT yet graduated. The substantive-rationale 3-part exception applies:

1. Substantive-rationale declaration: @neo-gpt's [GRADUATION_DEFERRED] STEP_BACK comment on #12062 (discussion-comment 17069393) explicitly identified OQ11 cap-raise as a standalone hot-fix that should be split out of the SSOT Epic graduation path. Operator (@tobiu) corroborated the 256K cap value (gemma-4-31b native context).
2. Inline divergence-matrix substance: see §"Avoided Traps / Divergence Matrix" below — 3 alternatives enumerated (status-quo cap / proposed raise / chunking-only) with falsifying sources per option.
3. Downstream amendment acknowledgment: this ticket is narrow config-only scope; sections that may need refresh post-Discussion #12062 graduation are flagged inline ("Out of Scope / Future-Refresh" section below).

Context

ai/config.template.mjs:125 + ai/config.mjs:125 set openAiCompatible.contextLimitTokens = 32768 with safeProcessingLimitTokens: undefined (defaults to Math.floor(32768 × 0.75) = 24,576 via DEFAULT_SAFE_FRACTION = 0.75 in ai/services/memory-core/helpers/ConsumerFrictionHelper.mjs:110). The default was sized for an earlier model-context era. gemma-4-31b-it (operator's current model per ai/config.mjs:120 model: 'gemma-4-31b-it') supports 256K (262,144) tokens natively. The cap is 8× smaller than the model's actual capacity.

Operator-surfaced empirical signal: "when the local gemma4 processing worked, my machine fans glew for 10 minutes. and this did not happen for a week or more." The disappearance maps to a combination of failure surfaces (full V-B-A in Discussion #12062 §2.4); this ticket addresses ONE of those surfaces — the invokeWithGuardrail size-cap pre-check in ai/services/graph/SemanticGraphExtractor.mjs:133-147.

The Problem

When invokeWithGuardrail runs estimatedTokens > safeProcessingLimitTokens check (24,576 tokens current), it aborts the LLM call BEFORE provider dispatch and returns null payload. SemanticGraphExtractor returns null, DreamService sees success = null, does NOT mark graphDigested: true, session is re-fetched next REM cycle → infinite silent-failure loop. No fans-glow because no LLM call ever happens for over-cap sessions.

Live distribution of recent session-payload sizes (sandman log, 2026-05-27):

Tally vs current cap: 5/10 over safe-band (guardrail abort); 4/10 over hard cap.

The Architectural Reality

• ai/config.template.mjs:118-127 — canonical openAiCompatible config block; defines contextLimitTokens + safeProcessingLimitTokens
• ai/config.mjs:107-116 — operator-overlay (currently in-sync with template per overlay-sync executed earlier this session)
• ai/config.template.mjs:521-523 + ai/config.mjs:478-480 — env-var bindings NEO_OPENAI_COMPATIBLE_CONTEXT_LIMIT_TOKENS + NEO_OPENAI_COMPATIBLE_SAFE_PROCESSING_LIMIT_TOKENS (operator-override path preserved)
• ai/services/memory-core/helpers/ConsumerFrictionHelper.mjs:110 — DEFAULT_SAFE_FRACTION = 0.75 (75% safe-band default)
• ai/services/memory-core/helpers/ConsumerFrictionHelper.mjs:329-377 — invokeWithGuardrail consumer; emits friction record + null result on size-cap exceed
• ai/services/graph/SemanticGraphExtractor.mjs:120-150 — consumer of the guardrail; computes consumerContextTokens + consumerSafeTokens from the openAiCompatible config block
• ai/services/graph/SemanticGraphExtractor.mjs:183-187 — if (!payload) return null on guardrail abort

gemma-4-31b-it published context: 256K tokens (262,144) (operator-confirmed; also applies to gemma-4-26B MoE per operator statement). Industry-standard 4 chars/token English avg → 262K tokens ≈ 1,048,576 chars; safe-band at 76% (200,000 tokens) ≈ 800,000 chars headroom.

The Fix

4-line config change in BOTH ai/config.template.mjs AND ai/config.mjs (now in-sync):

 openAiCompatible: {
     host                    : 'http://127.0.0.1:11434',
     model                   : 'gemma-4-31b-it',
     embeddingModel          : 'text-embedding-qwen3-embedding-8b',
     apiKey                  : '',
     unloadRetryCount        : 3,
     unloadRetryDelayMs      : 500,
-    contextLimitTokens      : 32768,
+    contextLimitTokens      : 262144,        // gemma-4-31b native 256K context
-    safeProcessingLimitTokens: undefined     // = 0.75 × 32768 = 24,576 (current default)
+    safeProcessingLimitTokens: 200000        // explicit; ~76% of cap; ~62K tokens headroom for system-prompt + response
}

Files touched:

• ai/config.template.mjs (canonical default — committed)
• ai/config.mjs (operator overlay — gitignored; commit MUST include this to satisfy MUST be in sync invariant per operator directive 2026-05-27 ~00:14Z)

Contract Ledger Matrix

Decision Record impact

• aligned-with ADR 0014 (cloud-deployment-topology) — uses the same configurable model substrate ADR 0014 established; no conflict
• none with any other ADRs

Acceptance Criteria

• AC1: ai/config.template.mjs updated — contextLimitTokens: 262144 + safeProcessingLimitTokens: 200000 with inline comment citing gemma-4 native context
• AC2: ai/config.mjs (operator overlay) updated to match — MUST be in sync invariant preserved per operator 2026-05-27 ~00:14Z directive
• AC3: V-B-A confirmation that aiConfig.openAiCompatible.contextLimitTokens === 262144 post-overlay-merge (Node REPL via node --check is insufficient; need runtime import check)
• AC4: Post-merge live ai:run-sandman invocation produces fans-glow signal on at least ONE undigested session (depends on companion #12059 routing fix landed first; AC4 deferred until #12061 merges)
• AC5: PR body cites the gemma-4 256K context-window evidence (operator V-B-A 2026-05-27 ~01:10Z) + the SemanticGraphExtractor.mjs:126 + ConsumerFrictionHelper.mjs:110 substrate anchors

Out of Scope / Future-Refresh

• OUT OF SCOPE — this ticket: structural chunking (OQ12 hierarchical-summarization in Discussion #12062 — Sub 7 of the SSOT Epic when it graduates); orchestrator-as-SSOT shape (Discussion #12062 graduation); silent-failure root-cause investigation across all 13 hypotheses (Sub 1 of the SSOT Epic); 5-axis observability primitive (Sub 2 of the SSOT Epic).
• REFRESH POST-DISCUSSION-#12062-GRADUATION: if the SSOT Epic graduates with a different config-substrate placement (e.g., per-consumer cap overrides instead of openAiCompatible-block defaults), this ticket's AC1/AC2 may need re-pointing. The cap value (262144) is operator-anchored and survives any substrate-placement reshape.

Avoided Traps / Divergence Matrix

Related

• Discussion #12062 — Orchestrator-as-SSOT for the REM (Sandman) Pipeline (parent ideation; OQ11 split this out per @neo-gpt STEP_BACK)
• #12059 — Fix Sandman graph provider routing (companion regression; @neo-gpt-owned; PR #12061)
• PR #11966 — original introduction of buildGraphProvider dispatch (the regression-introducing PR per Discussion #12062 §2.4 hypothesis #8)
• ADR 0014 cloud-deployment-topology (aligned-with)

Handoff Retrieval Hints

• query_raw_memories(query='contextLimitTokens safeProcessingLimitTokens gemma-4 256K context window cap')
• query_summaries(query='Sandman size-cap silent-failure invokeWithGuardrail')
• File anchors: ai/config.template.mjs:118-127 + ai/config.mjs:107-116 (config block); ai/services/memory-core/helpers/ConsumerFrictionHelper.mjs:110 (DEFAULT_SAFE_FRACTION); ai/services/graph/SemanticGraphExtractor.mjs:120-150 (consumer)
• Discussion #12062 §2.4.1 (the smoking-gun analysis section)

🤖 Generated with Claude Code