Repair session-summary completed-job drift

Context

The operator reported that a local manual workflow sync deferred behind the shared heavy-maintenance lease:

Deferred: heavy-maintenance lease held by 'summary' (reason='periodic-sweep:600000 pending-session-summary:472', pid=4193, acquiredAt=2026-06-18T08:10:27.031Z).

Read-only validation showed that the lease itself was real and later cleared: the orchestrator held summary from roughly 08:10:27 to 08:16:02 and then released it successfully. The bug is the backlog/no-progress substrate behind that lease, not the immediate CLI deferral.

Release classification: boardless. This is Agent OS reliability work, not automatically a release-board expansion unless the operator promotes it as release-blocking.

The Problem

The session-summary periodic sweep can keep reporting a large pending-session-summary backlog while making no progress.

Read-only probes against the live local graph DB showed:

• SummarizationJobs WHERE status='pending' returned 0.
• The current scheduler anti-join for getPendingSessionSummaryCount() is slow enough to hang on the live graph; SQLite EXPLAIN QUERY PLAN showed a full AGENT_MEMORY scan plus a correlated full SESSION_SUMMARY scan and temp distinct B-tree.
• A set-based CTE anti-join over distinct memory sessions and distinct summary sessions returned the same visible backlog (472) quickly.
• There are many live memory sessions where SummarizationJobs.status='completed' but no matching summary projection is visible to the scheduler. A targeted query found 150 sessions completed in the coordinator table but missing the SESSION_SUMMARY.properties.sessionId projection.
• Including the sibling graph shape where summary artifacts exist as SESSION nodes with properties.chromaId LIKE 'summary_%' still left 100+ completed jobs without a valid graph-visible summary artifact.

Net effect: findSessionsToSummarize() can select sessions because summary metadata/projection is missing or stale, but claimSummarizationJob() refuses them because the coordinator row is already completed. The child exits 0, the orchestrator logs success, and the periodic sweep repeats with little or no backlog reduction.

The Architectural Reality

• ai/daemons/orchestrator/scheduling/summary.mjs getPendingSessionSummaryCount() currently uses a correlated JSON NOT EXISTS anti-join against Nodes, which is null-safe but pathological on the live graph.
• ai/services/memory-core/SessionService.mjs findSessionsToSummarize() detects summary drift from Chroma memory/summary metadata, not from SummarizationJobs terminal state.
• SessionService.claimSummarizationJob() returns false for completed rows unconditionally.
• SessionService.completeSummarizationJob() marks the coordinator row completed after summarizeSession() returns a result, but historic/state-drift rows can exist where the coordinator is completed and the scheduler-visible summary projection is absent or malformed.
• ai/scripts/lifecycle/summarize-sessions.mjs writes key lifecycle lines to stdout, while ProcessSupervisorService captures child stderr only. That makes successful no-progress runs hard to diagnose from orchestrator.log.
• Tombstone logic for purged-session WAL/embed resurrection does not address this coordinator-vs-summary-artifact drift class.

The Fix

1. Rewrite getPendingSessionSummaryCount() as a set-based count over distinct memory sessions and distinct summary sessions, preserving null safety while avoiding the correlated full-scan plan.
2. Make the summary existence predicate match the graph shapes Neo has actually produced, or otherwise centralize it so scheduler count and summary repair agree on the same source of authority.
3. Add a repair path for SummarizationJobs.status='completed' when the summary artifact/projection is missing or stale, so completed coordinator rows cannot suppress required re-summarization forever.
4. Improve child-run observability so the orchestrator log shows pending drain, drift scan, skipped completed rows, and processed count for summary child runs.

Contract Ledger Matrix

Decision Record impact

none expected. This refines Memory Core/orchestrator reliability inside the existing heavy-maintenance and summary-state model. If implementation changes the task taxonomy or introduces a new durable state table, revisit ADR impact before PR.

Acceptance Criteria

• getPendingSessionSummaryCount() is set-based and remains null-safe when summary rows have NULL sessionId values.
• A completed SummarizationJobs row with a valid summary artifact remains terminal and is not reprocessed.
• A completed SummarizationJobs row with no valid summary artifact/projection can be repaired instead of being skipped forever.
• The summary periodic sweep does not hold the heavy-maintenance lease for a no-progress run caused solely by completed-without-artifact coordinator drift.
• Summary child logs expose enough information to distinguish 0 processed because no candidates, 0 processed because all candidates were already completed, and actual per-session progress.
• Existing pending/in-progress/failed summarization job semantics remain covered by tests.

Out of Scope

• Deleting or manually mutating local memory rows.
• Repairing Chroma vector corruption or query_summaries reachability; that was separately tracked under the summary-query corruption family.
• Reworking the heavy-maintenance lease taxonomy or KB-vs-MC backpressure policy.
• Changing provider/model timeouts unless a focused test shows this specific no-progress class still needs a watchdog after state repair.
• Changing purge tombstone semantics; the tombstone fix covers a different WAL/embed resurrection class.

Avoided Traps

• Blaming tombstones. The observed rows are coordinator/artifact drift, not purged-session WAL resurrection.
• Treating a successful child exit as progress. The child can exit 0 after skipping completed coordinator rows; the backlog can remain.
• Keeping the correlated anti-join because it is null-safe. Null safety is required, but the same property can be preserved with a set-based plan.
• Hiding the issue by excluding completed jobs from the count. That would reduce the metric while leaving missing summary artifacts unrepaired.

Duplicate / Claim Sweep

Live latest-open sweep: checked the latest 20 open issues immediately before creation on 2026-06-18. Relevant current open items included #13456, #13452, #13448, #13446, #13445, #13444, #13435, #13432, #13383, #13377, #13376, #13349, #13289, #13280, #13247, #13190, #13158, and #13144; no equivalent open ticket was present.

Targeted live GitHub GraphQL searches for pending-session-summary, SummarizationJobs SESSION_SUMMARY, and session summarization completed surfaced only broad parent/context issues (#12065, #12740, #10291, #12073, #12456) or no issue-specific match.

A2A in-flight sweep: checked recent 30 messages immediately before creation. No [lane-claim] or [lane-intent] overlapped this scope.

Semantic KB sweep: attempted ask_knowledge_base(query='open tickets session summarization backlog SummarizationJobs completed missing SESSION_SUMMARY pending-session-summary orchestrator summary lease', type='ticket'), but the local KB returned collections: requested resource could not be found. Fallback exact repo/content sweep covered resources/content/issues and resources/content/discussions for pending-session-summary, SummarizationJobs, SESSION_SUMMARY, session summarization, and heavy-maintenance lease; it surfaced adjacent closed/context tickets (#12830, #12943, #13358, #12450) but no equivalent open work item.

Related

• #12065 — parent area: Orchestrator-as-SSOT for the REM/Sandman pipeline.
• #12830 — adjacent closed investigation of corrupted/empty memories and slow synthesis stalls.
• #12943 — adjacent closed miniSummary backfill no-progress backoff fix.
• #13358 — adjacent closed KB-vs-MC backpressure split.
• #12450 — adjacent summary-query corruption/observability ticket.

Handoff Retrieval Hints:

• Retrieval Hint: "session-summary completed job missing SESSION_SUMMARY pending-session-summary 472 no-progress heavy-maintenance lease"
• Local probe shape: compare distinct AGENT_MEMORY.properties.sessionId against summary artifact projections, then join to SummarizationJobs.status='completed' to find completed-without-artifact drift.
• Log signature: Starting session summarization (periodic-sweep:600000 pending-session-summary:<N>) followed by session summarization completed successfully with little/no visible per-session progress.

Authored by GPT-5 (Codex Desktop) as @neo-gpt.