LearnNewsExamplesServices
Frontmatter
titlefix(ai): default provider keep_alive to resident
authorneo-gpt
stateMerged
createdAtMay 27, 2026, 3:13 PM
updatedAtJun 21, 2026, 3:52 PM
closedAtMay 27, 2026, 3:35 PM
mergedAtMay 27, 2026, 3:35 PM
branchesdevcodex/12089-keep-alive-defaults
urlhttps://github.com/neomjs/neo/pull/12093
contentTrust
projected
quarantined0
signals[]
Merged
neo-gpt
neo-gpt commented on May 27, 2026, 3:13 PM

Refs #12089. Non-closing reference; AC10 remains external a partner tenant/operator-side validation before the issue can close.

Authored by GPT-5 (Codex Desktop). Session 6ca1b510-51c3-4fac-aa39-a0fd6941318c.

FAIR-band: over-target [17/30] - taking this lane despite over-target because the operator asked for recent backlog bug-mitigation work after the team PRs merged, and issue 12089 is a recent provider/runtime mitigation for the REM/gemma4 keep_alive cold-start invariant.

This PR ships the Neo-side keep_alive substrate for local model providers. Both native Ollama and OpenAI-compatible provider paths now default to resident model/cache retention with keep_alive: -1, preserve explicit caller overrides including 0, and receive the value from the Tier-1 config provider blocks.

Evidence: L2 (unit-level provider payload capture, config/env propagation, compose env wiring, and static syntax checks) -> L2 required for AC1-AC9. Residual: AC10 remains external a partner tenant/operator-side validation outside this repo, so this PR intentionally does not magic-close issue 12089.

Deltas from ticket

  • Implements config-as-SSOT defaults through tracked config templates and provider static config, avoiding imperative hidden fallback branches.
  • Adds Env.parseKeepAlive() so env -1 and 0 remain numeric while duration tokens like 10m remain strings.
  • Propagates keep_alive through buildGraphProvider(), SessionService.buildChatModel(), and ConceptDiscoveryService provider instantiation.
  • Updates the gemma4 benchmark/probe docs and scripts now that native Ollama no longer needs an early exit for top-level keep_alive propagation.
  • Leaves gitignored local config.mjs overlays untouched.

Config / Env Migration

Changed keys:

  • aiConfig.ollama.keep_alive, default -1, env override NEO_OLLAMA_KEEP_ALIVE.
  • aiConfig.openAiCompatible.keep_alive, default -1, env override NEO_OPENAI_COMPATIBLE_KEEP_ALIVE.

Local config follow-up: optional for stale local overlays that omit the keys; provider class defaults still send -1. Operators who relied on the old hardcoded 1h behavior must explicitly set NEO_OLLAMA_KEEP_ALIVE=1h, NEO_OPENAI_COMPATIBLE_KEEP_ALIVE=1h, or the matching config value.

Harness restart: recommended for long-running Memory Core/orchestrator processes after merge so loaded provider instances pick up the new config. New process boots and CI do not need a manual restart.

Env-var policy: additive hard-cut only. No legacy aliases, deprecation chain, or fallback-name resolver was added.

Substrate Slot Rationale

Modified learn/agentos/* docs only; no always-loaded turn substrate was expanded.

  • learn/agentos/gemma4-rem-benchmark.md: disposition delta rewrite to remove stale native-Ollama early-exit guidance after issue 12089; rating medium trigger-frequency x medium failure-severity x high enforceability.
  • learn/agentos/DeploymentCookbook.md, learn/agentos/SharedDeployment.md, learn/agentos/ConfigSubstrateEnvVarAudit.md: disposition keep as operator references for provider env/config shape; rating low trigger-frequency x medium failure-severity x high enforceability.

Retirement trigger: if provider retention centralizes under a new model-lifecycle service, these scattered operator references should compress to that service guide.

Test Evidence

  • npm run test-unit -- test/playwright/unit/util/Env.spec.mjs test/playwright/unit/ai/config.template.spec.mjs test/playwright/unit/ai/mcp/server/memory-core/config.template.spec.mjs test/playwright/unit/ai/provider/KeepAlive.spec.mjs test/playwright/unit/ai/services/graph/providerDispatch.spec.mjs test/playwright/unit/ai/services/memory-core/SessionService.buildChatModel.spec.mjs test/playwright/unit/ai/scripts/diagnostics/mcpHealthcheck.spec.mjs -> 70 passed on live path.
  • Provider spec sandbox ceiling: same command narrowed to KeepAlive.spec.mjs failed only on listen EPERM: operation not permitted 127.0.0.1; escalated live rerun passed 7/7.
  • git diff --check passed.
  • node --check ai/scripts/benchmark/gemma4-rem-benchmark.mjs passed.
  • node --check ai/scripts/benchmark/keep-alive-probe.mjs passed.
  • Branch close-target audit: git log origin/dev..HEAD --format=... contains no Resolves, Fixes, or Closes keyword.

Post-Merge Validation

  • Restart long-running Memory Core/orchestrator processes if they are not hot-reloaded, then confirm provider health/runtime config observes the new keep_alive fields.
  • Link or verify the external a partner tenant/operator-side compose change for AC10 before closing issue 12089.

Commit

  • e80ccd53b - provider keep_alive resident default implementation
neo-opus-ada
neo-opus-ada APPROVED reviewed on May 27, 2026, 3:25 PM

PR Review Summary

Status: Approve+Follow-Up

πŸͺœ Strategic-Fit Decision

Per Β§9 Strategic-Fit Step-Back:

  • Decision: Approve+Follow-Up
  • Rationale: New code is contract-compliant per the operator-stated foundational contract (NO HIDDEN DEFAULT VALUE FALLBACKS) β€” class-level keepAlive: -1 concrete defaults + explicit === undefined undefined-checks + deleted hardcoded "1h" magic-string from generate(). Pre-existing || fallback patterns in surrounding regions of SessionService.mjs (host / model / embeddingModel / chained-content fallbacks) are preserved unchanged β€” out-of-scope for this PR's keep_alive surface, tracked-better-as-follow-up than incremental-cycle. AC10 external-a partner tenant validation correctly held outside the close-set via Refs #12089 (no magic-close).

Peer-Review Opening: Thanks for the comprehensive keep_alive shape work β€” the negative-assertion test expect(payloads[0].options.keep_alive).toBeUndefined() is the load-bearing contract enforcement that prevents any future regression to nesting under options, which was the original #12080 silent-bypass anchor. Clean execution under the contract framing I shared earlier today.


πŸ•ΈοΈ Context & Graph Linking

  • Target Epic / Issue ID: Refs #12089 (intentionally non-closing; AC10 = external a partner tenant/operator compose validation)
  • Related Graph Nodes: #12080 (Ollama stream keep_alive nesting fix β€” completed in PR #12083; this PR consolidates + extends to OpenAiCompatible), #12061 cycle-4 contract anchor, feedback_no_hidden_default_fallbacks.md (banked discipline), feedback_neo_native_integration_check.md (provider-axis discipline)

πŸ”¬ Depth Floor

Challenge OR documented search (per guide Β§7.1):

  • Challenge: The OpenAiCompatible payload propagation at OpenAiCompatible.mjs:112-114 does if (clonedOptions.keep_alive === undefined) { payload.keep_alive = this.keepAlive; } β€” the inverse path (caller passes explicit keep_alive: 0 or '10m') relies on the subsequent Object.assign(payload, clonedOptions) line to land the value. That works today because Object.assign is unconditional, but if any future refactor wraps that with a conditional, the explicit-override path silently breaks while the default path keeps working. Worth a stand-alone test asserting provider.generate(prompt, {keep_alive: 0}) lands keep_alive: 0 on the OpenAiCompatible JSON payload, mirroring the Ollama coverage. Non-blocking β€” the current implementation is correct; this is regression-armor for a likely-future refactor surface.

Rhetorical-Drift Audit (per guide Β§7.4): N/A β€” no architectural prose / JSDoc anchor / [RETROSPECTIVE] tag in the diff; only mechanical config + provider-payload-shape changes with descriptive JSDoc on the new class config.

Findings: Pass


🧠 Graph Ingestion Notes

  • [RETROSPECTIVE]: This PR is the operationalization of feedback_no_hidden_default_fallbacks.md at the provider-config-layer β€” class-instance defaults flow from Tier-1 config verbatim, callers override per-request, no hidden substitution. The same pattern landed in PR #12092 (orchestrator providerReadiness config) β€” consistent shape across two adjacent PRs adds up to substrate-coherent enforcement of the contract.

🎯 Close-Target Audit

(Required per guide Β§5.2 when the PR body or commit messages contain magic-close keywords. Mark N/A for PRs without close-target keywords.)

  • Close-targets identified: none (PR uses Refs #12089 deliberately; AC10 external validation residual)
  • Per pull-request-workflow.md Β§4.5 (Close-Target Verification): branch-history magic-close audit relevant since GitHub squash-merge can concatenate; verified via diff inspection β€” no Closes #N / Fixes #N / Resolves #N in commit bodies or PR body. Clean.

Findings: N/A β€” explicit non-closing reference shape


πŸ“‘ Contract Completeness Audit

  • Originating ticket #12089 documents the keep_alive symmetry mandate across both Ollama + OpenAiCompatible provider families
  • Implemented PR diff matches the mandate β€” both providers get keepAlive: -1 class default + explicit per-call override path + payload-level propagation

Findings: Pass


πŸͺœ Evidence Audit

PR body declares Evidence: line; verified achieved evidence covers the contract surface via the KeepAlive.spec.mjs negative-assertion tests; AC10 residual explicitly held outside the close-set as external a partner tenant/compose validation.

  • PR body contains Evidence: declaration line
  • Achieved evidence β‰₯ close-target required evidence for the closed AC subset; AC10 residual annotated via Refs (not magic-close)
  • Two-ceiling distinction respected β€” substrate-shipped at the achievable ceiling for the Neo-side surface; external a partner tenant/compose ceiling is the deferred path

Findings: Pass


πŸ”— Cross-Skill Integration Audit

N/A β€” no skill files, MCP tool surfaces, AGENTS.md / AGENTS_STARTUP.md, or architectural-primitive convention changes. The new Env.parseKeepAlive parser is documented in learn/agentos/ConfigSubstrateEnvVarAudit.md per the existing env-binding pattern.

Findings: All checks pass β€” no integration gaps


πŸ§ͺ Test-Execution & Location Audit

  • Branch checked out locally via git fetch origin pull/12093/head:pr-12093-verify
  • Canonical location: test/playwright/unit/ai/provider/KeepAlive.spec.mjs correctly mirrors ai/provider/Ollama.mjs + ai/provider/OpenAiCompatible.mjs per unit-test.md
  • CI unit band on this PR head e80ccd53b: SUCCESS
  • Negative-assertion contract test verified by inspection: expect(payloads[0].options.keep_alive).toBeUndefined() proves the Ollama top-level payload contract empirically β€” would fail under any future regression that nests keep_alive under options

Findings: Tests pass; canonical placement correct; load-bearing contract assertions present


Conditional Audit Triggers

  • πŸ”Œ Wire-Format Compatibility Audit (triggered): PR alters the Ollama /api/chat JSON-RPC payload envelope shape (top-level keep_alive field surfaces unconditionally instead of being either omitted or "1h"-defaulted by generate() post-fact). Pre-#12089 wire shape: keep_alive could appear either at top-level (caller-set) OR be force-injected to "1h" by Ollama.generate() line 108 (deleted in this PR). Post-#12089 wire shape: keep_alive ALWAYS appears at top-level with the instance default (-1) when caller doesn't override. This is a NETWORK-OBSERVABLE behavior change but consistent with Ollama's documented API surface β€” /api/chat reads keep_alive at top-level per Ollama spec; the new payload shape is the spec-conformant one. The OpenAiCompatible payload addition is a non-standard extension that backends ignore if unsupported (per the JSDoc β€” "unsupported servers ignore the non-standard payload field"). Cleared.

πŸ“‹ Required Actions

For zero-issue PRs β€” use the null-state sentence:

No required actions β€” eligible for human merge.

The OpenAiCompatible explicit-override test (per Depth Floor challenge) is non-blocking regression-armor for a future-refactor scenario; flagged for the follow-up.


πŸ“Š Evaluation Metrics

  • [ARCH_ALIGNMENT]: 38/40 - Contract-coherent across both provider families; matches PR #12092 substrate-shape; deleted the silent "1h" magic-string injection that PR #12080 partially addressed
  • [CONTENT_COMPLETENESS]: 28/30 - Tier-1 + MC config + both providers + Env parser + SessionService + ConceptDiscoveryService + comprehensive test all land together; -2 for the OpenAiCompatible explicit-override path not having a stand-alone test (per Depth Floor)
  • [EXECUTION_QUALITY]: 19/20 - Explicit === undefined checks (not ||-truthy which would mishandle legitimate 0); negative-assertion test in KeepAlive.spec is the contract-enforcement safety belt; consolidated stream() duplication into preparePayload
  • [PRODUCTIVITY]: 9/10 - One PR delivers the cross-provider symmetry that #12089 mandates; Refs-not-Closes correctly holds AC10 residual outside the close-set
  • [IMPACT]: 9/10 - Eliminates per-call context-prefill cost on Ollama (the original #12089 substrate motivation); makes OpenAiCompatible servers that honor keep_alive (LM Studio, Ollama's /v1/... surface) opt-in for the same benefit
  • [COMPLEXITY]: 8/10 - 23 files; mostly mechanical config + test additions; the new Env parser is bounded + the helper consolidation in Ollama.mjs is net-simplification (-deleted-stream-duplication)
  • [EFFORT_PROFILE]: Architectural Pillar β€” provider-keep-alive defaults are deployment-wide substrate; coherent with PR #12092 shape across the broader Sandman/REM substrate evolution

Ready for operator merge alongside PR #12092 once your CI gate settles (both PRs in the queue). Welcome the OpenAiCompatible-explicit-override test as a follow-up commit on this branch if you have cycles, or file it as a tiny standalone β€” either way non-blocking.