Ollama.stream() issues its fetch with no timeout and no abort signal

Context

Surfaced while measuring whether ollama abandons in-flight inference on client disconnect (#16706). That measurement was later falsified — see the struck claim below — but the question it raised stands on its own: is there any path of ours that can never stop waiting, and would therefore hang a consumer indefinitely?

There is one. It is not the cause of any live incident, and this ticket says so up front so nobody re-derives that hope.

The Problem

Ollama.stream() issues its request as a bare fetch with no cancellation surface at all:

const response = await fetch(`${this.host}/api/chat`, {
    method : 'POST',
    headers: {'Content-Type': 'application/json'},
    body   : JSON.stringify(payload)
});

No signal, no timeout, no abort path. Its sibling OpenAiCompatible.stream() does the opposite:

...(controller ? {signal: controller.signal} : {})

And Ollama.generate() — the same class, the adjacent method — documents the property this method lacks: *"Upstream cancellation signal; when it aborts, the in-flight request is destroyed (parity with OpenAiCompatible)."* The parity is asserted for generate() and silently absent in stream().

Corrected 2026-08-10 08:3xZ, on @neo-gpt's intake. This ticket originally named Ollama.chat(). There is no such method — ai/provider/Ollama.mjs exposes generate(), embed() and stream(). The quoted JSDoc and the timeout idiom both belong to generate(). A reviewer following the original text would have gone looking for a method that does not exist.

Why it matters — restated, because the original justification was falsified the same day.

⛔ The struck claim. This ticket said: "ollama computes for exactly as long as the client stays connected." That is false, and #16853 measured it false under control: an early-aborted request left the runner at ~400 % CPU with zero established sockets, persisting after every client was stopped. Closing the socket proves the client released the transport. It proves nothing about whether the provider is still computing.

I had published that same "ollama cancels on disconnect" generalisation once already, from a control that differed from production on three axes. This ticket inherited it. It is struck here rather than quietly rewritten, because the original wording was the entire stated justification.

What this ticket actually fixes, bounded to what a caller can control: Ollama.stream() has no way to stop waiting. With no timeout and no abort signal, a caller that issues it has no bounded exit — the consumer hangs until the process dies, and no upstream deadline or user cancellation can reach it. Every other method in the class bounds its own wait; this one cannot.

Explicitly NOT claimed, and owned by #16853: that bounding the caller releases the provider's work. It does not, and the fix must not be described as if it does. #16849 is caller/transport-bounded; server-work uncertainty after a disconnect is #16853's subject. A repair here that reported "cancelled" as "settled" would reintroduce the exact defect #16853 exists to remove.

Not the cause of any live incident — stated explicitly

Ollama.stream() has no production caller. Every caller of .stream() in ai/ is either OpenAiCompatible's own internal use or one of two scripts under ai/scripts/benchmark/ (gemma4-rem-benchmark.mjs, keep-alive-probe.mjs).

It fits the pegged-runner evidence beautifully — which is exactly why the caller check ran before the claim did. A mechanism that fits the evidence and reaches no production line is a story, not a diagnosis. This is filed as the defect it is: a provider-asymmetry and a latent hazard, not an incident cause.

The practical risk is real but bounded: the two benchmark scripts are the kind of thing an operator runs against a deployed plane and walks away from, and a stalled one has no self-bound.

The Architectural Reality

• ai/provider/Ollama.mjs — async *stream(input, options = {}), the bare fetch call.
• Same file, generate() and embed() — both resolve timeoutMs, both honour options.signal, both destroy the request on timeout. The timeout/abort idiom to copy is already in the class twice.
• ai/provider/OpenAiCompatible.mjs — async *stream() arms signal: controller.signal. This is the parity target.

The Fix

Give Ollama.stream() the cancellation surface its siblings already have: resolve timeoutMs the way generate() does, honour an upstream options.signal, and abort the in-flight request when either fires. No new configuration — generate()'s existing timeout resolution is the pattern. Bound the CALLER's wait; do not report the abort as provider settlement (#16853).

Contract Ledger Matrix

Decision Record impact: none — provider-layer parity repair, no config leaves.

Acceptance Criteria

• Ollama.stream() aborts its in-flight request when its resolved timeout elapses, proven against a server that never responds.
• Ollama.stream() aborts when an upstream options.signal fires, matching OpenAiCompatible.stream().
• The socket is observed to close on abort — not merely that the generator threw. A pooled keep-alive agent can decouple "the client gave up" from "the connection closed", and a caller that abandons without closing leaves the transport held. Note what this AC does NOT establish: #16853 measured zero established sockets coexisting with a ~400 % runner, so a closed socket bounds the CLIENT and says nothing about the provider.
• Mutation control: removing the abort wiring reddens the stall spec and no other.
• generate() / embed() behaviour is unchanged — their existing specs stay green.

Out of Scope

• Any claim that this explains a pegged runner on a deployed plane. It has no production caller; see above.
• The disconnect-cancellation question itself. measured and settled: ollama cancels — struck: #16853 falsified it. Whether the provider keeps computing after a disconnect is that ticket's subject, and this one must not assume either answer.
• Adding a caller for stream(), or removing it.

Avoided Traps

• Filing this as an incident cause. It fits the evidence perfectly and reaches no live line. The fit is what makes the caller check mandatory rather than optional.
• Folding only the paragraph a reviewer points at. The first correction pass fixed the cited section and left the same falsified claim standing in Context, an AC and Out of Scope — a partially folded ticket reads as corrected while still asserting the thing it retracted. Enumerate every CLAIM that rests on the falsified premise, not every step.
• Asserting the generator throwing is sufficient. On a pooled keep-alive socket, a client that gives up without closing the connection leaves the provider computing — so the AC is written against the socket, not the exception.

Related

• #16706 — the epic whose disconnect measurement surfaced this.
• #16830, #14154 — the live provider lanes; this is deliberately not claimed as their mechanism.

Live latest-open sweep: latest 20 open issues checked at 2026-08-10T00:2xZ; no equivalent found.

Structure map: N/A — modifies ai/provider/Ollama.mjs in place.

Origin Session ID: d8332b13-5d97-4839-ac11-d2de4602a989

Retrieval Hint: query_raw_memories("Ollama stream fetch no timeout no abort signal parity OpenAiCompatible benchmark-only caller")