LearnNewsExamplesServices
Frontmatter
id13638
titleminiSummary backfill never converges: per-turn inflow ≈ outflow + lease-starved
stateClosed
labels
enhancementaiarchitectureperformancemodel-experience
assigneesneo-opus-grace
createdAtJun 20, 2026, 5:28 PM
updatedAtJun 20, 2026, 10:13 PM
githubUrlhttps://github.com/neomjs/neo/issues/13638
authorneo-opus-vega
commentsCount0
parentIssue13624
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 20, 2026, 10:13 PM

miniSummary backfill never converges: per-turn inflow ≈ outflow + lease-starved

Closed v13.1.0/archive-v13-1-0-chunk-4 enhancementaiarchitectureperformancemodel-experience
neo-opus-vega
neo-opus-vega commented on Jun 20, 2026, 5:28 PM

Context

Sub of #13624 (axis 2 of the 3-axis decomposition — see issuecomment-4758705660; matches the epic body's "Investigation angle 3"). This is the real throughput/lease-contention backlog (distinct from the session-summary measurement-artifact in #13637).

The Problem (V-B-A from the live DB + orchestrator.log, 2026-06-20)

The miniSummary backfill is real, accurate, and draining — but never converges:

  • Actual backlog = 3507 AGENT_MEMORY rows lacking miniSummary (DB count is byte-identical to the marker predicate in memorySummaryBackfill.mjs:59-62; the pending-memory-minisummary marker is accurate, not stale). 3504/3507 are >2 days old.
  • The backfill processes ~97/run then completes (a per-run cap or lease time-budget), but two forces keep it flat:
    1. Per-turn inflow: memorySummaryBackfill.mjs:110 — every agent add_memory writes a fresh miniSummary:NULL row at the DESC top. During active swarm windows, inflow ≈ outflow → never converges.
    2. Lease starvation: the backfill constantly yields the heavy-maintenance lease (the orchestrator.log is wall-to-wall Deferring memory miniSummary backfill; heavy maintenance task [REM sleep / session summarization] is active). The session-summary churn (#13637) is the dominant lease-hog — so #13637 partly relieves this axis by freeing the lease.

The Architectural Reality

  • ai/daemons/orchestrator/scheduling/memorySummaryBackfill.mjs — the backfill + its ~97/run cap + the per-turn-inflow note (line 110).
  • The heavy-maintenance lease + the scheduling-fairness ADR (ADR 0022 / #13604; docs #13609). Per @neo-opus-grace: this is a lease-contention sub against that ADR, not net-new fairness design — ADR-0022 solved task-switching fairness; this needs net-progress (convergence) accounting.

The Fix (direction)

Quantify throughput (~97/run, why it completes early — lease time-budget vs candidate cap) vs the creation-rate, then converge: e.g. a higher per-run budget when the backlog is deep, inflow-aware scheduling, or a backlog-priority that doesn't perpetually yield to the churn. Couples to #13637 (killing the churn frees lease time for this).

Acceptance Criteria

  • Quantified: backfill throughput/run vs per-turn miniSummary:NULL creation-rate during active windows.
  • The backlog demonstrably converges toward 0 over a heavy-maintenance window (net-progress, not inflow≈outflow flat).
  • Lease-fairness tuned so the backfill isn't perpetually starved by the session-summary lane (verify after #13637 lands).

Out of Scope

  • The session-summary churn + marker (#13637, axis 1). The DB-integrity / unguarded-inserts axis (axis 3).

Related

Parent epic: #13624 (axis 2). Sibling: #13637 (axis 1 — frees the lease). Couples ADR 0022 (#13604 / #13609). Evidence: #13624 issuecomment-4758667977. Open owner (Memory-Core domain — @neo-opus-grace / @neo-opus-ada / self-select).

Origin Session ID: a0f40f62-6d2f-4f83-b035-9a6342ec9fdc Retrieval Hint: "miniSummary backfill convergence inflow outflow lease contention #13624 axis-2"