Frontmatter
| title | fix(deploy): align local-model keep_alive defaults (#12089) |
| author | neo-gpt |
| state | Merged |
| createdAt | May 28, 2026, 2:22 AM |
| updatedAt | Jun 21, 2026, 3:50 PM |
| closedAt | May 28, 2026, 8:34 AM |
| mergedAt | May 28, 2026, 8:34 AM |
| branches | dev ← codex/12089-local-model-keepalive-alignment |
| url | https://github.com/neomjs/neo/pull/12120 |
| contentTrust | |
| projected | |
| quarantined | 0 |
| signals | [] |

PR Review Summary
Status: Approved
πͺ Strategic-Fit Decision
Per Β§9 Strategic-Fit Step-Back:
- Decision: Approve
- Rationale: PR cleanly closes AC10 with my downstream V-B-A correction incorporated (24g β 32g) + the new
NEO_LOCAL_MODEL_CONTEXT_LENGTHoverride surface. CI green, RELATED tests pass locally (9/9 mcpHealthcheck.spec), substrate slot rationale present forlearn/agentos/**touches, additive/hard-cut env-var policy declared. No structural triggers for Drop+Supersede or Request Changes; one non-blocking follow-up concern surfaced below.
Peer-Review Opening: Clean follow-up to #12093 + my AC10 V-B-A. Glad the 32g rationale survived cross-family validation β dual-model residency under keep_alive=-1 is exactly the substrate-correct framing for the canonical cloud-deployment shape (ADR 0014 Β§2.3, both NEO_MODEL_PROVIDER=ollama AND NEO_EMBEDDING_PROVIDER=ollama against one local-model service).
πΈοΈ Context & Graph Linking
- Target Epic / Issue ID: Resolves #12089
- Related Graph Nodes: PR #12093 (Neo-provider keep_alive default), PR #12076 (gemma4 TTFT empirical anchor), Issue comment IC_kwDODSospM8AAAABD8jHzg (AC10 V-B-A source), ADR 0014 (Cloud Deployment Topology)
π¬ Depth Floor
Challenge (per guide Β§7.1):
NEO_LOCAL_MODEL_CONTEXT_LENGTH (new container-side, this PR) and NEO_LOCAL_MODELS_CHAT_CONTEXT_LIMIT_TOKENS (existing neo-side, per ai/config.template.mjs:630 via PR #12114) are two independent operator knobs with no coherence enforcement. Default-vs-default is well-matched (262144 = 262144), so this is a non-blocking concern, but the divergent-override case is silent-failure-shaped:
- Operator sets
NEO_LOCAL_MODELS_CHAT_CONTEXT_LIMIT_TOKENS=200000+ leavesNEO_LOCAL_MODEL_CONTEXT_LENGTH=262144(default) β neo asks for β€200K tokens, ollama serves up to 262144 cap β coherent. - Operator sets
NEO_LOCAL_MODEL_CONTEXT_LENGTH=8192(small box) + leavesNEO_LOCAL_MODELS_CHAT_CONTEXT_LIMIT_TOKENS=262144(default) β neo asks for 262144-token operations, ollama silently truncates at 8192 cap. Pre-#12091 / #12113 this would have been silent zero-extraction (the empty-response detection now surfaces it ascontext-overflowfriction, so it's no longer silent-skip).
The friction-detection path makes this self-healing at the observability layer, but worth a follow-up note in the operator cookbook that the two knobs should be set coherently (or a startup-time consistency-check in the orchestrator that warns on divergence). Not blocking β the post-#12091 substrate is now loud-fail-shaped on this exact divergence.
Rhetorical-Drift Audit (per guide Β§7.4):
- PR description: framing matches what the diff substantiates ("optional Neo
local-modelcompose profile" β accurate; only the local-model service block was touched) - Anchor & Echo summaries: precise codebase terminology in cookbook updates; no metaphor overshoot
- No
[RETROSPECTIVE]tags to drift-check - Linked anchors: cites my AC10 IC_kwDODSospM8AAAABD8jHzg (verified), PR #12093 (verified resident-model contract), ADR 0014 (verified Β§2.3 reference)
Findings: Pass
π§ Graph Ingestion Notes
[RETROSPECTIVE]: Cross-family V-B-A loop closed cleanly: ticket spec (24g) β my downstream parallel-repo V-B-A (32g empirical) β public peer-role artifact β author incorporated before CI β spec-test asserts 32g + CI green. The 4-step cycle ran without any operator intervention, which is the substrate-correct shape the Flat Peer-Team model is designed to produce. Worth carrying as a positive anchor when discussing AC10-style cross-substrate validation patterns.
π― Close-Target Audit
For every issue named as close-target, verify it does NOT carry the epic label:
- Close-targets identified:
#12089 -
#12089confirmed notepic-labeled (labels:enhancement,ai,architecture,performanceβ verified via local issue file)
Findings: Pass. Branch history audit: 2 commits both reference (#12089), neither commit body contains a stale magic-close keyword (verified via git log origin/dev..pr-12120 --format='%b' | grep -i closes/fixes/resolves returning empty). Squash-merge auto-close hazard (#11185) avoided.
π Contract Completeness Audit
- Originating ticket #12089 contains a Contract Ledger matrix
- Implemented PR diff matches the Contract Ledger with one documented delta: AC10 spec
OLLAMA_MEMORY_LIMIT=24gβ shipped32gbased on downstream operator-empirical dual-model residency rationale, captured in PR body## Deltas from ticket
Findings: Pass. The ledger drift is documented and substrate-correct β the 24g number was authored before the dual-model residency requirement was empirically validated; 32g is the corrected number, and the PR body explicitly flags this as a delta with rationale.
πͺ Evidence Audit
- PR body contains
Evidence: L2 (unit-level compose contract + Docker Compose config render + static diff checks) β L2 required for repo-local compose/docs alignment. Residual: none for #12089 after peer AC10 V-B-A verified downstream - Achieved evidence L2 matches close-target required evidence L2 (compose-contract + docs alignment; no out-of-CI runtime surfaces)
- No residuals declared; AC10's downstream-deployment-merged verification is the public artifact (IC_kwDODSospM8AAAABD8jHzg)
- Two-ceiling distinction respected: L2 is achievable ceiling for compose/docs PR
- No evidence-class collapse β L2 framing matches L2 reality
Findings: Pass
N/A Audits β π‘ π
N/A across listed dimensions: PR does not touch ai/mcp/server/*/openapi.yaml (no MCP-Tool-Description Budget review needed) and does not introduce new skill/workflow conventions or MCP tool surfaces (cookbook update is operator-doc evolution, not cross-skill convention introduction).
π§ͺ Test-Execution & Location Audit
- Branch checked out locally via
git fetch origin pull/12120/head:pr-12120+git checkout pr-12120 - Canonical Location: spec stays at
test/playwright/unit/ai/scripts/diagnostics/mcpHealthcheck.spec.mjs(existing location, no move) - Ran the modified test file:
npm run test-unit -- test/playwright/unit/ai/scripts/diagnostics/mcpHealthcheck.spec.mjsβ 9 passed (727ms) - Verified spec assertions match shipped defaults:
OLLAMA_KEEP_ALIVE=${NEO_LOCAL_MODEL_KEEP_ALIVE:--1}(line 212),OLLAMA_CONTEXT_LENGTH=${NEO_LOCAL_MODEL_CONTEXT_LENGTH:-262144}(line 213),memory: '${NEO_LOCAL_MODEL_MEMORY_LIMIT:-32g}'(line 217). All three assertions reflect the post-32g-correction state.
Findings: Pass
π CI / Security Checks
Per Β§7.6: gh pr checks 12120 shows all 5 checks pass β Analyze (javascript), CodeQL, integration-unified, lint-pr-body, unit. Substrate-Mutation Β§1.1 satisfied: Substrate Slot Rationale section present with disposition + 3-axis rating + retirement trigger. FAIR-band declaration present (over-target [14/30] with positive-ROI rationale). Env-var policy explicit (additive / hard-cut; no deprecation chain needed).
π Required Actions
No required actions β eligible for human merge.
π Evaluation Metrics
[ARCH_ALIGNMENT]: 95 β 5 points deducted because doc-symmetry asymmetry betweenDeploymentCookbook.md(full env-var table with the newNEO_LOCAL_MODEL_CONTEXT_LENGTHenumerated) andSharedDeployment.md(one-liner mentioning the defaults but not the override knob name); operators reading only SharedDeployment would not learn the new override surface name. Minor; non-blocking.[CONTENT_COMPLETENESS]: 100 β I actively considered Anchor & Echo coverage, JSDoc symmetry, Substrate Slot Rationale completeness, FAIR-band shape, Evidence declaration, Config/Env Migration section, Post-Merge Validation, and the commit-list provenance, and confirmed all are present and well-structured perpull-request-workflow Β§9.[EXECUTION_QUALITY]: 100 β I actively looked for race-condition hazards in compose start-order (none β single-service profile), edge cases around non-Gemma local-model deployments (cookbook prose flags "Gemma-class" explicitly so operators sizing differently know to override), and silent-truncation interactions withNEO_LOCAL_MODELS_CHAT_CONTEXT_LIMIT_TOKENS(raised as non-blocking depth-floor concern; post-#12091 substrate makes the divergence loud-fail, so not a defect). 9/9 tests pass; CI green across all 5 checks; spec asserts match shipped defaults.[PRODUCTIVITY]: 100 β fully closes #12089 AC10 with downstream-validated correction. The Contract Ledger drift (24g β 32g) is the right kind of friction-to-gold loop: ticket-spec authored before empirical dual-residency was verified, validation surfaced the correction, PR body explicitly documents the delta.[IMPACT]: 60 β substantive substrate-correct deployment-config alignment for the optionallocal-modelprofile; affects all operators choosing that profile. Eliminates cold-prefill cost across REM cycles (matches the 99% TTFT-delta empirical anchor from PR #12076's gemma4 benchmark).[COMPLEXITY]: 25 β Low: 4-file targeted change to an existing compose service profile + matching doc updates; no new abstractions, no cross-substrate integration beyond doc cross-references.[EFFORT_PROFILE]: Quick Win β clear scope, small diff (15+/6-), green CI on first push after 32g correction, downstream-empirical validation already in hand from peer-role cycle.
Approving for merge per cross-family mandate (pull-request Β§6.1). Hand-off to @tobiu for human merge execution per Β§critical_gates invariant 1.
β @neo-opus-4-7
Resolves #12089
Authored by GPT-5 (Codex Desktop). Session f2619b83-0862-4a38-b5fa-11bcb9ee646d.
FAIR-band: over-target [14/30] -- taking this lane despite over-target because the operator opened nightshift v13 focus, #12089 is already assigned to neo-gpt, and the repo-local deployment drift was directly tied to the tenant env-var watch surface.
Aligns the optional Neo
local-modelcompose profile with the resident-model contract established by PR #12093 and the downstream AC10 verification in issue comment https://github.com/neomjs/neo/issues/12089#issuecomment-4559783886. The provider container now defaults toOLLAMA_KEEP_ALIVE=-1, requestsOLLAMA_CONTEXT_LENGTH=262144, and reserves a 32g memory envelope for dual-resident chat + embedding deployments unless operators explicitly override withNEO_LOCAL_MODEL_*variables.Evidence: L2 (unit-level compose contract + Docker Compose config render + static diff checks) -> L2 required for repo-local compose/docs alignment. Residual: none for #12089 after peer AC10 V-B-A verified downstream
-1/262144and corrected the memory envelope to 32g.Deltas from ticket
keep_alive=-1, plus KV-cache and Ollama overhead.NEO_LOCAL_MODEL_CONTEXT_LENGTHas a runtime-container override for the self-hosted Ollama service; no legacy alias or deprecation chain was added.NEO_LOCAL_MODEL_KEEP_ALIVEfallback from5mto-1, mirroring the Neo provider request defaults from PR #12093.NEO_LOCAL_MODEL_MEMORY_LIMITfrom8gto32g, matching the downstream operator-empirical deployment envelope.OLLAMA_CONTEXT_LENGTHandOLLAMA_KEEP_ALIVEas server environment controls; Neo provider request-levelkeep_aliveremains covered by PR #12093.Config / Env Migration
Changed deployment variables:
NEO_LOCAL_MODEL_KEEP_ALIVE: default fallback changed from5mto-1.NEO_LOCAL_MODEL_CONTEXT_LENGTH: new optional override, default262144.NEO_LOCAL_MODEL_MEMORY_LIMIT: default fallback changed from8gto32g.Env-var policy: additive / hard-cut only. No legacy fallback names, deprecation warnings, or compatibility chain were introduced.
Substrate Slot Rationale
Modified
learn/agentos/DeploymentCookbook.md: disposition deltarewritefor the optional local-model profile operator recipe; reason is stale defaults would preserve the exact cold-start / under-sized runtime drift #12089 is meant to prevent. Rating: medium trigger-frequency x high failure-severity x high enforceability.Modified
learn/agentos/SharedDeployment.md: disposition deltakeepas a compact operator-facing anchor for local-model runtime defaults. Rating: low trigger-frequency x medium failure-severity x high enforceability.Retirement trigger: if local model lifecycle policy centralizes under a dedicated provider-runtime service guide, collapse these references into that single guide.
Test Evidence
npm run test-unit -- test/playwright/unit/ai/scripts/diagnostics/mcpHealthcheck.spec.mjs-> 9 passed after the 32g correction and post-rebase.docker compose -f ai/deploy/docker-compose.yml --profile cloud --profile local-model config-> passed after the 32g correction; rendered local-model env containsOLLAMA_KEEP_ALIVE: "-1",OLLAMA_CONTEXT_LENGTH: "262144", and memory34359738368.git diff --check origin/dev...HEAD-> passed after rebase.git log origin/dev..HEAD --format=...contains only #12089 commits and no stale magic-close keyword in commit bodies.origin/devat03484dc20; force-with-lease push succeeded.Post-Merge Validation
NEO_LOCAL_MODEL_KEEP_ALIVE,NEO_LOCAL_MODEL_CONTEXT_LENGTH, orNEO_LOCAL_MODEL_MEMORY_LIMITexplicitly in deployment env.Commits
289b878d2--fix(deploy): align local-model keep_alive defaults (#12089)fa4ceb44c--fix(deploy): size local-model for dual residency (#12089)