LearnNewsExamplesServices
Frontmatter
id17045
titleOrchestrator embed path can hang without a deadline and starve the tenant-sync scheduler
stateClosed
labels
bugaiagent-os
assignees[]
createdAt11:32 AM
updatedAt11:45 AM
githubUrlhttps://github.com/neomjs/neo/issues/17045
authorneo-opus-vega
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAt11:45 AM

Orchestrator embed path can hang without a deadline and starve the tenant-sync scheduler

Closed Backlog/active-chunk-15 bugaiagent-os
neo-opus-vega
neo-opus-vega commented on 11:32 AM

Context

Live external-plane evidence (2026-08-13, deployment-state snapshots via the MC surface): the orchestrator's tenant-repo-sync scheduler froze for 90+ minutes (lastRunAt pinned at 07:56:31 on a 30-minute cadence / 60-second sweep) while the orchestrator container reported healthy — its healthcheck watches the state-file mtime, which the deployment-state bridge thread keeps fresh, so a dead scheduler inside a live process is invisible to Docker.

The wedge location was proven by probe divergence: the orchestrator's own embedding recovery probe timed out (EMBEDDING_PROBE_TIMEOUT) while kb-server's identical probe against the same provider passed. One process's serial embed dispatch was stuck; the provider was fine.

Timeline: a sync attempt failed instantly at 07:56 (KB_VECTOR_EMBED_FAILED recorded in the same second the run started — an unserviceable call, not a slow one); the self-heal system restarted the embedding provider container at 08:10; the orchestrator's queue never recovered afterward. A candidate mechanism — a request whose socket died in the provider restart without surfacing an error — should have been released by the 60-minute batch ceiling, and was not, which points at a call on this path that carries no deadline at all (or an await that a transport error never rejects).

This is the third layer of the same disease class in one incident family: engine (abandoned work never dies) → client process queue (serial dispatch starved by a hung head) → scheduler (starved by the queue). Each layer's lesson is identical: every call on the embed path must carry a deadline, and every abort must propagate to whatever is awaiting it.

The Fix (shape)

  1. Audit every await on the orchestrator's embedding dispatch path (recovery probe, tenant-sync ingest calls, WAL-adjacent embeds) for missing or unreachable deadlines; wrap the transport so socket death and container restarts reject promptly.
  2. The serial dispatch queue gets a queue-head watchdog: a head older than the maximum legitimate ceiling is failed and evicted, loudly, rather than starving the process forever.
  3. The scheduler's liveness becomes observable independently of the state-file mtime (the bridge keeping the file fresh must not mask a dead sync loop) — a lastRunAt-age field on the snapshot plus a degraded healthcheck contribution when it exceeds the cadence by a bound.

Acceptance Criteria

  • A fixture proving a never-resolving provider call cannot freeze the scheduler: the queue head is evicted at its bound and the next sweep runs.
  • A fixture proving provider-container death mid-request rejects the in-flight call promptly (no reliance on OS-level socket timeouts).
  • Scheduler staleness (lastRunAt age > cadence × N) is visible on the deployment-state snapshot and degrades the orchestrator's health contribution.
  • No behavior change for calls that complete within their deadlines.

Related

#16972 (batch retry sizing — sibling, distinct defect) · #16706 (external-plane recovery epic) · #17044 (self-heal futility circuit breaker — same incident family) · #17042 (retrospective mechanics)

Origin Session ID: 944f029a-1fab-46a1-9d11-a9fadd003db8