LearnNewsExamplesServices
Frontmatter
id12080
titlePatch `Ollama.stream()` to inject top-level `keep_alive` (mirror `Ollama.generate()` line 108)
stateClosed
labels
enhancementaiarchitecture
assigneesneo-gpt
createdAtMay 27, 2026, 5:04 AM
updatedAtJun 7, 2026, 7:16 PM
githubUrlhttps://github.com/neomjs/neo/issues/12080
authorneo-opus-ada
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtMay 28, 2026, 2:53 AM

Patch Ollama.stream() to inject top-level keep_alive (mirror Ollama.generate() line 108)

Closed v13.0.0/archive-v13-0-0-chunk-14 enhancementaiarchitecture
neo-opus-ada
neo-opus-ada commented on May 27, 2026, 5:04 AM

Parent context

Follow-up surfaced by @neo-gpt PR #12076 cycle-2 review (Epic #12065 Sub 8 / #12074). The keep-alive-probe.mjs benchmark cannot characterize native Ollama keep_alive control because of an SDK-level payload-shape gap; this ticket owns the SDK fix.

V-B-A finding (banked from PR #12076 cycle-2)

ai/provider/Ollama.mjs:

Line Observation
91-92 (in preparePayload()) Arbitrary remaining options are nested under payload.options (e.g. provider.stream(messages, {keep_alive: '1h'})payload.options.keep_alive = '1h')
108 (in generate()) payload.keep_alive = "1h" is set TOP-LEVEL after preparePayload() returns — bypasses the option-nesting
271 (in stream()) Serializes the unmodified preparePayload() result — no top-level keep_alive injection

Native Ollama /api/chat reads keep_alive at TOP LEVEL of the request body. The nested payload.options.keep_alive is silently ignored by Ollama's REST API. This makes caller-supplied keep_alive via provider.stream() unobservable on the native Ollama path.

Prescription

Mirror Ollama.generate() line 108's pattern in Ollama.stream():

async *stream(input, options = {}) {
    const payload = this.preparePayload(input, options, true);
    // Promote top-level keep_alive (mirror of generate() line 108) so streaming
    // callers can control KV-cache retention. Without this, keep_alive nested
    // under payload.options is silently ignored by Ollama's /api/chat.
    if (options.keep_alive !== undefined) {
        payload.keep_alive = options.keep_alive;
        if (payload.options) delete payload.options.keep_alive;
    }
    // ... existing fetch logic
}

(Final shape may also extract keep_alive from payload.options to remove the nested duplicate.)

Acceptance Criteria

  • AC1: Ollama.stream() promotes caller-supplied keep_alive from nested payload.options to top-level payload.keep_alive before serialization
  • AC2: Unit test covering top-level keep_alive injection (mock fetch verifies payload.keep_alive is set when caller passes options.keep_alive)
  • AC3: ai/scripts/benchmark/keep-alive-probe.mjs Ollama early-exit (added in PR #12076 cycle-2) is REVERTED — probe can now characterize native Ollama keep_alive control empirically
  • AC4: learn/agentos/gemma4-rem-benchmark.md "Substrate V-B-A findings" table row for Ollama.mjs:91-92 updated to reflect the fix
  • AC5: Verify OpenAiCompatible.stream() does NOT have the same nesting bug — current cycle-2 V-B-A shows OpenAiCompatible.preparePayload() line 105 already uses Object.assign(payload, clonedOptions) so top-level IS the natural shape. AC5 may close as "no-op confirmed" once verified

Avoided Traps

  • Don't expand scope to a generic provider-options-promotion pattern — keep_alive is the documented Ollama-specific request-body field; other payload.options.* fields are correctly Ollama's options model-parameters object (temperature, top_p, etc.)
  • Don't change Ollama.generate() line 108 — it already does the right thing; this ticket only adds parity to the stream() path
  • Don't bundle with broader provider refactor — Epic #12065 has Sub 3 (executeRemCycle) which may touch provider lifecycle; this ticket is a focused SDK fix

Related

  • Epic #12065 — Orchestrator-as-SSOT REM pipeline
  • #12074 / PR #12076 — gemma4 benchmark Sub 8 (this ticket unblocks AC3 native-Ollama keep_alive characterization)
  • @neo-gpt cycle-2 review on PR #12076 (2026-05-27) — original V-B-A surface
  • learn/agentos/gemma4-rem-benchmark.md — substrate V-B-A findings table
tobiu referenced in commit 831683c - "fix(ai): promote Ollama stream keep_alive (#12080) (#12083) on May 27, 2026, 2:19 PM