LearnNewsExamplesServices
Frontmatter
id13590
titleSession-summary backlog never settles: miniSummary prerequisite + metric reconcile
stateClosed
labels
bugaiarchitectureperformance
assigneesneo-gpt
createdAtJun 20, 2026, 5:56 AM
updatedAtJun 20, 2026, 11:43 AM
githubUrlhttps://github.com/neomjs/neo/issues/13590
authorneo-opus-grace
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 20, 2026, 11:43 AM

Session-summary backlog never settles: miniSummary prerequisite + metric reconcile

Closed v13.1.0/archive-v13-1-0-chunk-4 bugaiarchitectureperformance
neo-opus-grace
neo-opus-grace commented on Jun 20, 2026, 5:56 AM

Context

Operator-reported, 2026-06-20: after #13579 fixed the count-mismatch pagination re-loop and #13586 / PR #13593 landed the heavy-maintenance fair picker, the session-summary lane can still fail to reach a trustworthy quiet state.

The original body framed this primarily as "dead-letter poison sessions." That framing was corrected in the issue thread on 2026-06-20 and is now folded into the close contract here.

Corrected Diagnosis

Primary correction: ordinary count drift cannot persist once summarizeSession() writes a summary with the full paginated memoryCount; the next findSessionsToSummarize() pass sees matching counts and self-heals. The old "count-mismatch retries forever" premise was falsified by source and by #13579.

The true recurring path is narrower:

  1. summarizeSession() raw synthesis can be skipped or time out.
  2. The degraded fallback depends on per-turn miniSummary rows from memory-summary-backfill.
  3. If the summary lane outruns or starves that backfill, degraded fallback either uses lossy 280-char raw snippets or returns null.
  4. A session with no successful summary remains a drift candidate.

For sessions too large even after all mini-summaries exist, hierarchical summarization belongs to #12073. Dead-lettering is a final backstop for truly corrupt / unprocessable sessions, not the first fix for sessions missing their mini-summary prerequisite.

Current Source Reality

  • ai/daemons/orchestrator/scheduling/registry.mjs: summary and memory-summary-backfill are independent heavy tasks; summary.dependencies is still [].
  • ai/daemons/orchestrator/scheduling/pipeline.mjs: #13593 added staleness-aware priority across heavy tasks; it did not add a semantic prerequisite from mini-summary backfill to session summary.
  • ai/services/memory-core/SessionService.mjs: findSessionsToSummarize() still derives candidates from Chroma drift; summarizeSession() falls back to getSessionMiniSummaries() and then truncated raw snippets.
  • ai/daemons/orchestrator/scheduling/summary.mjs: pending-session-summary:N is graph-backed via getPendingSessionSummaryCount(), while the drainer is Chroma-backed. That count can remain decoupled from what the summary child can actually drain.

Contract Ledger

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
summary periodic drift scheduling registry.mjs, summary.mjs, memorySummaryBackfill.mjs, pipeline.mjs Periodic drift summary must not run ahead of drainable memory-summary-backfill work that provides its degraded-summary prerequisite. The exact implementation may be a due-task guard or explicit dependency predicate, but it must be source-tested and must not block sunset handovers or explicit pending SummarizationJobs markers without evidence. If backfill is in a documented no-progress backoff, summary may proceed and use the best available fallback. Existing scheduling JSDoc plus tests. Unit tests proving summary yields while drainable mini-summary backlog exists, and still fires for handovers / pending markers where appropriate.
pending-session-summary:N trigger reason summary.mjs:getPendingSessionSummaryCount(), SessionService.findSessionsToSummarize() The reported backlog must distinguish drainable summary drift from graph-projection residue so operators can tell when the lane is actually quiet. Fail soft to null / omitted reason when the required substrate is unavailable; do not report a misleading fixed count as proof of drainable work. Function JSDoc. Unit coverage with graph-summary projection divergence and a clear expected trigger reason.
Terminal unprocessable-session state SessionService.summarizeSession(), SummarizationJobs, #12073 Dead-lettering is a backstop after prerequisites are present or proven unavailable, not a shortcut for sessions starved of mini-summaries. Preserve failed job state and emit observable failure; do not silently exclude recoverable sessions. Function JSDoc / follow-up ticket if the backstop is split. Unit coverage when this slice implements it; otherwise explicit follow-up and out-of-scope note.

Acceptance Criteria

  • Periodic summary drift scheduling is dependency-aware: when miniSummary backfill has drainable pending work and is not in no-progress backoff, periodic summary yields instead of consuming the heavy lane first. Unit-tested.
  • Sunset-handover and explicit pending SummarizationJobs behavior is preserved or intentionally documented with evidence. Unit-tested.
  • pending-session-summary reporting no longer presents a graph-only anti-join residue as indistinguishable drainable Chroma drift. Unit-tested against graph / Chroma projection divergence or an equivalent seam.
  • No implementation dead-letters a session merely because its miniSummary prerequisite was starved. Dead-letter behavior, if included, is a backstop only and emits operator-visible evidence.
  • Post-merge: the live orchestrator reaches a quiet / explained state for summary backlog instead of repeating the same fixed pending-session-summary:N without progress.

Out of Scope

  • Fair heavy-task picker priority / staleness logic — delivered by #13586 / PR #13593.
  • Bounded summary lease-hold / max sessions per sweep — #13592.
  • Hierarchical summarization for sessions too long even after mini-summaries — #12073.
  • Per-provider throughput tuning.

Related

  • #13586 / PR #13593 — heavy-maintenance fair picker, merged.
  • #13592 — bounded summary lease-hold so the picker gets more release boundaries.
  • #12073 — hierarchical summarization for truly oversize sessions.
  • #13579 — full pagination count fix; falsifies the old count-mismatch-forever diagnosis.

Decision Record impact

none for this correction if implemented as scheduler / observability behavior within the existing orchestrator and Memory Core contracts. If the implementation introduces a new durable dead-letter state or changes task taxonomy semantics, file or update the relevant decision record before PR.

Release classification: post-v13 Agent OS stability hardening.

Retrieval Hint: "summary backlog settles miniSummary prerequisite pending-session-summary graph Chroma divergence" / summary.dependencies / getPendingSessionSummaryCount / findSessionsToSummarize