LearnNewsExamplesServices
Frontmatter
id10484
titleOpenAiCompatible stream fallback for non-SSE JSON chat completions
stateClosed
labels
bugai
assigneesneo-gpt
createdAtApr 29, 2026, 12:05 PM
updatedAtJun 3, 2026, 11:58 AM
githubUrlhttps://github.com/neomjs/neo/issues/10484
authorneo-gemini-pro
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 3, 2026, 11:58 AM

OpenAiCompatible stream fallback for non-SSE JSON chat completions

Closed Backlog/active-chunk-8 bugai
neo-gemini-pro
neo-gemini-pro commented on Apr 29, 2026, 12:05 PM

Context

This ticket was originally bundled as LM Studio Tri-Vector pipeline crash + SQLite chunking refactor. Current V-B-A shows that bundle is stale:

  • The SQLite chunking / SQLITE_IN_CLAUSE_BATCH_SIZE work was already delivered by #10463 / PR #10464 and #10484-linked PR #10486.
  • Empty-provider-output detection and downstream graph-extractor friction routing were later improved by #12091 / #12113-era work.
  • The remaining provider-contract gap is still real: current ai/provider/OpenAiCompatible.mjs requests stream: true and only yields SSE choices[0].delta.content. If an OpenAI-compatible server replies with a non-SSE JSON chat-completions payload, e.g. choices[0].message.content, the stream parser yields no content.

Problem

OpenAI-compatible providers can return either streaming SSE chunks or a plain JSON chat-completions object. Neo currently handles the SSE delta path only. For local providers such as LM Studio / llama.cpp that may respond with a monolithic JSON object under response_format: {type: "json_object"} or degraded streaming support, this can collapse into an empty generated payload.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback / Edge Case Docs Evidence
OpenAiCompatible.stream() SSE path Current provider contract Continue yielding choices[0].delta.content from data: SSE frames. Malformed individual SSE frames remain ignored as today. JSDoc if behavior clarified Existing behavior preserved by focused provider test
OpenAiCompatible.stream() non-SSE JSON path OpenAI-compatible chat-completions response shape If the response body is a plain JSON object, yield choices[0].message.content when present. Empty / missing content yields no chunks; invalid JSON should not corrupt valid SSE behavior. Provider JSDoc New unit test with JSON response fixture
OpenAiCompatible.generate() aggregate result Existing generate() delegates to stream() Aggregates the non-SSE yielded content exactly like SSE content and returns {content, raw.message.content}. Provider errors still throw through. No extra docs unless JSDoc changes Focused generate/stream coverage
SQLite chunking / bridge-daemon GraphLog insertions #10463 / PR #10464 / PR #10486 Already resolved; not part of this residual ticket. None. Do not re-open bridge-daemon chunking under this ticket. N/A Current source/tests + closed PRs

Acceptance Criteria

  • OpenAiCompatible.stream() preserves existing SSE delta.content behavior.
  • OpenAiCompatible.stream() yields choices[0].message.content for non-SSE JSON chat-completions responses.
  • OpenAiCompatible.generate() aggregates that non-SSE content through its existing stream delegation.
  • Focused unit coverage exercises both SSE and non-SSE JSON response shapes.
  • No SQLite chunking or bridge-daemon scope is included; that stale slice remains closed by prior PRs.

Related

Origin Session ID: original #10484 context; body narrowed by @neo-gpt on 2026-06-03 after live source and PR verification.

tobiu referenced in commit 7e1bbc0 - "fix(ai): externalize SQLite chunking limit to prevent parameter overflow (#10484) (#10486) on Apr 29, 2026, 2:53 PM
tobiu referenced in commit 6ddfc4f - "fix(ai): handle non-sse OpenAI-compatible JSON (#10484) (#12428) on Jun 3, 2026, 11:58 AM
tobiu closed this issue on Jun 3, 2026, 11:58 AM