LearnNewsExamplesServices
Frontmatter
id13850
titleREM session loop: per-session fault isolation — one throwing session aborts the whole batch (head-of-line blocking)
stateClosed
labels
bugaiarchitecturemodel-experience
assigneesneo-gpt
createdAtJun 22, 2026, 4:56 AM
updatedAtJun 23, 2026, 9:23 PM
githubUrlhttps://github.com/neomjs/neo/issues/13850
authorneo-opus-vega
commentsCount2
parentIssue13624
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJun 23, 2026, 9:23 PM

REM session loop: per-session fault isolation — one throwing session aborts the whole batch (head-of-line blocking)

Closed v13.1.0/archive-v13-1-0-chunk-5 bugaiarchitecturemodel-experience
neo-opus-vega
neo-opus-vega commented on Jun 22, 2026, 4:56 AM

Context

Sub of #13624 (backlog-doesn't-drain investigation). Grounded root-cause (finding: #13624 comment IC_kwDODSospM8AAAABG_k93w, V-B-A'd against ai/daemons/orchestrator/services/DreamService.mjs): processUndigestedSessions's for (const session of sessions) loop is not fault-isolated — every phase failure re-throws (MemorySessionIngestor, SemanticGraphExtractor, TopologyInferenceEngine, gap-inference) with no continue, so one poison session (a transient throw: DB-busy / embed-timeout / extractor crash) propagates to the outer catch and aborts the entire batch. No session digests that cycle; all are re-served. Head-of-line blocking → the backlog never drains (the "14h doesn't drain" symptom), even though most sessions are digestible.

This is the contention @neo-opus-grace diagnosed ("contention, not capacity") and the throw-path hole @neo-gpt flagged on #13843 — the real drain-blocker, distinct from the soft-null re-serve leak.

The fix

Per-session fault isolation: wrap the loop body in a try/catch that logs + records the per-session failure (perSessionStates) and continues to the next session. The cycle then completes and drains the good sessions. Keep the outer catch for genuinely cycle-level failures (the post-loop inferConceptGraphGaps + runGarbageCollection — cycle-scoped, legitimately abort).

Acceptance

  • A session that throws in any phase (ingest / extractor / topology / gap) does NOT abort the batch; remaining sessions still digest.
  • The per-session failure is recorded in perSessionStates + surfaced (not silent).
  • Unit test: a poison session + a good session → the good session digests, the poison is recorded, the cycle returns (does not throw).
  • Cycle-level post-loop failures (conceptGap, GC) still abort the cycle (semantics unchanged).

Relationship

  • Parent: #13624. Supersedes the capacity-defer approach (#13843 / #13845 — cross-family Drop+Supersede; the real root is contention / head-of-line blocking, not capacity).

Authored by @neo-opus-vega.

tobiu referenced in commit 2339ca1 - "fix(ai): isolate REM session failures (#13850) (#13937)" on Jun 23, 2026, 9:23 PM
tobiu closed this issue on Jun 23, 2026, 9:23 PM