Switch REM chat-model to gemma-4-26b-a4b MoE (no-think + json_schema)

Context

The Golden Path froze for 18 days (#13750) and sessions stopped digesting into the graph. A cluster of tickets attacked symptoms — #13851/#13852 (serve the model at the right context), #13835/#13843/#13845 (stop re-serving un-digestible sessions). This session ran a full Verify-Before-Assert benchmark sweep (direct LM Studio :1234, orchestrator off) and found the root cause is the model choice, not context or capacity.

Release classification: post-release (post-v13 agent-harness / REM pipeline; not release-blocking) → boardless.

The Problem

gemma-4-31b-it (dense) has a catastrophic cold-prefill cost on this hardware: measured ~47s prefill for a ~9k-token session, ~137s at ~30k tokens. Per-session REM extraction pays cold prefill on every unique session, so heavy sessions exceeded the safe band / timeout → choke → null → never digested → Golden Path starved. #13851 correctly caught one layer (resident at 4096 ≠ configured 131072), but even at 131072 the dense 31B is too slow — context-serving is necessary but not sufficient.

gemma-4-26b-a4b is a Mixture-of-Experts model (~4B active params). Measured same-prompt, same-machine, both loaded at 131072:

• Prefill 3.0s vs 31B's 46.7s (~15×); ~5s vs ~137s at 30k tokens (~25×). This is the actual choke fix.
• It is a thinking model (emits hidden reasoning_content). reasoning_effort:"none" disables it → ~2× faster (summary 13.0s→6.0s) with zero measured quality loss (summary self-score 95 vs 95; extraction identical node/edge counts).
• Quality parity with the 31B on real 25-turn session summaries AND real tri-vector extraction (both valid, accurate, no hallucination).

The Architectural Reality

• Model leaf: aiConfig.openAiCompatible.model (config leaf, env NEO_OPENAI_COMPATIBLE_MODEL; ADR 0019 reactive Provider SSOT). Default gemma-4-31b-it. The exact LM Studio identifier is google/gemma-4-26b-a4b — note gemma-4-26-a4b (missing the b) is rejected as an invalid identifier. Tested host LM Studio :1234; confirm openAiCompatible.host against live config at implementation.
• reasoning_effort is a per-call request param — verified. Default always-off; keep per-call as a documented lever.
• Provider: ai/provider/OpenAiCompatible.mjs:97-98 emits response_format:{type:'json_object'} when responseMimeType==='application/json'. LM Studio rejects json_object ("must be 'json_schema' or 'text'"). Fix: emit response_format:{type:'json_schema', json_schema:{…caller schema…}}.
• Callers needing structured output: SessionService.summarizeSession (buildSummaryPrompt) and SemanticGraphExtractor.executeTriVectorExtraction (L74-127 systemInstruction + nested a2a/session_artifact/graph schema). With json_schema, the extractor's repair-retry loop (L226) and relaxed-default loop (L233) stop firing on the happy path — output is valid/fence-free/schema-correct first-try. Verified working even with LM Studio's GUI "Structured Output" toggle OFF → API-controllable, no deploy prerequisite (contrast flash-attention, which is GUI-only).

The Fix

1. aiConfig.openAiCompatible.model default → google/gemma-4-26b-a4b (ai/config.template.mjs + JSDoc).
2. Wire reasoning_effort:"none" for the local-model summary + extraction calls (default always-off; per-call lever documented in JSDoc).
3. OpenAiCompatible.mjs: replace the json_object branch with json_schema — accept a caller-supplied schema and pass it through; keep a text fallback when no schema is supplied. SessionService + SemanticGraphExtractor pass their schemas.

Contract Ledger Matrix

Decision Record impact

aligned-with ADR 0019 — model + reasoning_effort are config-leaf/per-call values resolved at the use-site; the provider reads them, never re-implements. Provider json_schema is a structured-output capability addition + latent-bug fix (json_object is rejected by LM Studio). Empirical input to the in-progress target-architecture ADR Discussion #13846 (REM organism); does NOT depend on or graduate from it.

Acceptance Criteria

• openAiCompatible.model default is google/gemma-4-26b-a4b; summary + extraction run against it.
• Local summary + extraction calls send reasoning_effort:"none"; the per-call lever is documented in JSDoc.
• OpenAiCompatible.mjs emits response_format:{type:'json_schema',…} with the caller's schema; the json_object path no longer errors against LM Studio; text fallback retained.
• SessionService.summarizeSession + SemanticGraphExtractor.executeTriVectorExtraction pass their schemas; extraction yields valid schema-correct output without the repair-retry loop firing on the happy path.
• (post-merge, live) A real heavy session digests end-to-end with no choke, verified via a live REM run once the orchestrator is up.

Out of Scope

• Ollama / cloud chat-model parity — the MoE's ollama tag differs, and ollama's structured-output (native format mechanism) + no-think toggle need their own verification → separate linked follow-up ticket (do not fold in).
• Tri-vector graph richness — extraction is sparse (~4 nodes/session) in BOTH think and no-think modes → a prompt-completeness lane affecting all models, separate from this switch.
• Orchestrator serving lifecycle (#13851/#13852) — still valid (now serve the MoE at the right context); this ticket changes WHICH model + how JSON is requested, not the serving mechanism.

Avoided Traps

• "It's only the context window." #13851's 4096-vs-131072 is real but not the whole root — the dense 31B is too slow even at 131072 (47s prefill). Verified.
• "The 4-bit MoE produces unreliable JSON." A streamed-output glitch looked like malformed keys; non-stream verification showed clean keys — it was stream serialization / the test harness, not the model.
• "Keep thinking for summaries." Measured: thinking gave zero quality lift for summary OR extraction at ~2× cost → always-off is the evidence-backed default (the per-call lever stays for a future hard-summary test).

Related

• Parent epic: #12740 (per-task model routing / local-first provider defaults + cost-safety) — concrete realization (SUB).
• Reframes (necessary-but-not-sufficient): #13851, #13852.
• Reduces criticality of (fewer chokes → keep as safety net): #13835, #13843, #13845.
• Downstream symptom: #13750 (Golden Path freeze), #13837/#13844 (candidate-pool).
• Adjacent: #12742 (default summarization to a local provider).
• Empirical input to: #13846 (target-architecture ADR Discussion).
• Follow-up to file: ollama/cloud chat-model parity.

Origin Session ID

2439c28b-245e-425a-85a7-ca7ee4aa0336

Handoff Retrieval Hints

• query_summaries: "gemma-4-26b-a4b MoE no-think json_schema REM extraction summary prefill"
• Live latest-open sweep: checked latest 25 open issues at 2026-06-22T10:01Z; no equivalent (closest #13851/#13852 are serving-layer, not model-choice). A2A claim sweep + KB semantic sweep: no in-flight claim, no duplicate ticket.