LearnNewsExamplesServices
Frontmatter
id14036
titleEmbedding-provider freeze/runaway detection (per-call duration + CPU)
stateClosed
labels
enhancementaiarchitecture
assigneesneo-gpt
createdAtJun 26, 2026, 1:48 AM
updatedAtJun 26, 2026, 10:10 AM
githubUrlhttps://github.com/neomjs/neo/issues/14036
authorneo-opus-vega
commentsCount1
parentIssue14039
subIssues
14052 Bound native Ollama embedding calls with timeouts
subIssuesCompleted1
subIssuesTotal1
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJun 26, 2026, 10:10 AM
milestonev13.1

Embedding-provider freeze/runaway detection (per-call duration + CPU)

Closed v13.1.0/archive-v13-1-0-chunk-6 enhancementaiarchitecture
neo-opus-vega
neo-opus-vega commented on Jun 26, 2026, 1:48 AM

Context

An over-cap embedding input (a too-large source/document before chunking existed) was observed to freeze the embedding provider — pegging multiple ollama worker CPU cores at 100% with no completion — which both stalled the deferred-embed WAL drain (a metadata-without-vector pathway, see #14027/#13999) and burned the host. The pre-call mitigations since added (input guards #13928, context-truncation #13944, oversized chunking #14000) prevent the known over-cap cause, but there is no signal that catches a frozen/runaway embed call from any cause while it is happening.

The Problem

Provider readiness checks (#13851 loaded-context, #13950 load-aware, residency diagnostics) verify the model is up before a call. Input guards reject known-oversized inputs before a call. Neither detects a call that has started and is hung/runaway (over-cap that slipped a guard, a provider bug, a pathological input). A single hung embed call can stall the WAL drain indefinitely and saturate CPU — silently, until a downstream symptom (failed backup, coverage drop) surfaces days later.

The Architectural Reality

  • Shared embedding path: TextEmbeddingService.embedTexts (used by KB + MC); the embed-daemon drains the add_memory WAL through it (#12838/#12840).
  • #13551 (embed-drain watchdog) alarms on a stalled WAL backlog — a downstream/aggregate signal; this ticket is the per-call upstream signal that explains why the drain stalled.
  • ADR-0025 diagnostics detect container CPU saturation, but not per-embed-call duration.

The Fix

A per-embed-call watchdog: bound each embed call with a duration timeout + emit a signal when a call exceeds a threshold (and/or when embedding-provider CPU saturates with no throughput). On timeout: abort the call, route the input to the unrecoverable/retry path (fail-loud), and emit an ADR-0025 diagnosis. Prevents one hung call from stalling the whole drain.

Acceptance Criteria

  • Each embed call is bounded by a configurable duration timeout (AiConfig leaf); a call exceeding it is aborted and the input routed to a fail-loud retry/unrecoverable path (not left hanging).
  • A runaway/timed-out call emits an ADR-0025 diagnosis (escalate-with-diagnosis), not a silent stall.
  • Optional: provider CPU-saturation-with-no-throughput is surfaced as a contention signal.
  • A hung embed call cannot indefinitely stall the embed-daemon WAL drain (one bad call is isolated).
  • Unit coverage: a simulated hung embed call triggers the timeout + escalation, drain continues.

Out of Scope

  • Pre-call input guards / chunking (#13928 / #14000 — complementary, already landed).
  • The WAL-drain backlog watchdog (#13551 — the downstream/aggregate complement).

Related

  • #14027 / #13999 — the freeze→WAL-drain-stall→metadata-without-vector mechanism this catches at the source.
  • #13551 — embed-drain backlog watchdog (downstream complement).
  • #14026 — data-integrity detect-signal (sibling).
  • #13928 / #13944 / #14000 — pre-call over-cap mitigations.

Origin Session ID: 58acd7a6-778e-4fe6-b85a-4569802ac57b

Handoff Retrieval Hints: query_raw_memories("embedding provider freeze runaway per-call duration timeout WAL drain stall CPU saturation"); anchors: TextEmbeddingService.embedTexts, embed-daemon WAL drain (#12838/#12840), #13551.

Authored-by: @neo-opus-vega (Vega, Claude Opus 4.8)

tobiu referenced in commit 0478a57 - "fix(ai): bound embedding batch requests (#14036) (#14047)" on Jun 26, 2026, 10:10 AM
tobiu closed this issue on Jun 26, 2026, 10:10 AM