LearnNewsExamplesServices
Frontmatter
id12809
titleOrchestrator summary backlog count is capped at the fetch limit (blind to true depth)
stateClosed
labels
enhancementai
assigneesneo-opus-grace
createdAtJun 9, 2026, 4:24 AM
updatedAtJun 9, 2026, 10:04 AM
githubUrlhttps://github.com/neomjs/neo/issues/12809
authorneo-opus-grace
commentsCount0
parentIssue12740
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 9, 2026, 10:04 AM

Orchestrator summary backlog count is capped at the fetch limit (blind to true depth)

neo-opus-grace
neo-opus-grace commented on Jun 9, 2026, 4:24 AM

Context

@tobiu's biggest-friction call (prio-0 v13): "the orchestrator messages don't tell us how many session + mini summaries are LEFT in the backlog — we are blind." V-B-A'd to the line: both summary schedulers cap their pending count at the fetch LIMIT (50), so the logged trigger reason (pending-memory-minisummary:50, pending-summarization:50) is the fetch limit, not the true backlog depth. With 50 fetched out of e.g. 1842 pending, the log always reads :50 → no way to tell if the backlog is draining or growing.

The Problem

  • ai/daemons/orchestrator/scheduling/memorySummaryBackfill.mjsgetPendingMemorySummaryBackfillJobs(db, {limit=50}); the trigger reason uses pendingJobs.length (≤50).
  • ai/daemons/orchestrator/scheduling/summary.mjsgetPendingSummarizationJobs(db, {limit=50}); same, pending-summarization:${pendingJobs.length} (≤50).

The fetched IDs exist only for the existence-check (.length); the supervised child re-queries its own batch. So the reason's number is purely the capped fetch length.

The Fix

Add an uncapped COUNT(*) to each scheduler (getPendingMemorySummaryBackfillCount, getPendingSummarizationCount), fail-soft, and thread it into the trigger reason (pending-X:${trueDepth}). The reason is what the orchestrator logs, so the operator sees the real backlog + whether it is draining. No output-shape change (backward-compatible); the fetched-IDs existence-check is unchanged.

Decision Record impact

none — orchestrator scheduling/logging; no ADR governs it.

Acceptance Criteria

  • Both schedulers expose an uncapped pending COUNT, fail-soft (null when the table is unavailable).
  • The trigger reason reflects the true backlog depth, not the fetch limit (seed > 50 pending → reason shows the true count).
  • Output trigger shape unchanged (existing consumers/tests unaffected).
  • Tests verify the true-depth reason in both schedulers.

Out of Scope

  • The 4s-timeout drain-fix (#12804 / PR #12805) + run-budget (#12746 / PR #12802). Lease-starvation (heavy tasks always running, mutually deferring). Per-item progress reporting beyond the backlog count.

Related

Live latest-open sweep: per the 2026-06-09T01:50:00Z latest-20 sweep (+ #12804/#12805 filed since, both mine); no observability-count dup.

Authored by Claude Opus 4.8 (Claude Code)

tobiu referenced in commit 45220b3 - "feat(orchestrator): report true summary backlog depth, not the fetch limit (#12809) (#12810) on Jun 9, 2026, 10:04 AM
tobiu closed this issue on Jun 9, 2026, 10:04 AM