LearnNewsExamplesServices
Frontmatter
id12673
titleBackfill miniSummary for pre-existing memories — most-recent-first batches
stateClosed
labels
enhancementaitesting
assigneesneo-gpt
createdAtJun 7, 2026, 4:15 AM
updatedAtJun 7, 2026, 6:25 PM
githubUrlhttps://github.com/neomjs/neo/issues/12673
authorneo-opus-grace
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 7, 2026, 6:25 PM

Backfill miniSummary for pre-existing memories — most-recent-first batches

Closed v13.0.0/archive-v13-0-0-chunk-16 enhancementaitesting
neo-opus-grace
neo-opus-grace commented on Jun 7, 2026, 4:15 AM

Context

PR #12671 added an inline best-effort tweet-summary (miniSummary) at add_memory for query_recent_turns — but it is forward-looking only. The ~15k pre-existing AGENT_MEMORY nodes (plus any new ones written while the summarizer was transiently unavailable — AC8 is fail-soft) have no miniSummary and fall back to raw content. This ticket backfills them, mirroring the existing session-summarization batch service. Operator-flagged 2026-06-07.

Live latest-open sweep: checked latest 15 open issues at 2026-06-07T02:14:43Z; no equivalent found.

The Problem

query_recent_turns(detail:'summary') returns the compact miniSummary when present, else falls back to raw/truncated content (fail-soft by design — #12671 AC8). For the recovery feed to be token-efficient across an agent's history (not just turns since the feature shipped), the existing un-summarized memories need backfilling. Roughly 15k memories lack a summary today.

The Architectural Reality — the precedent to mirror

Session summaries are backfilled by a batch service; this ticket mirrors it for memories:

  • ai/scripts/lifecycle/summarize-sessions.mjs — the spawned lifecycle child that summarizes session batches.
  • ai/daemons/orchestrator/taskDefinitions.mjs:86 — the Orchestrator task definition that schedules it.
  • ai/daemons/orchestrator/scheduling/summary.mjs (getPendingSummarizationJobs, limit-based) — the batch finder.
  • SessionService.mjssummarizationBatchLimit, drift detection, most-recent-first.

The pieces this reuses already exist (shipped in #12671):

  • MemoryService.buildMiniSummary({prompt, response}) — the per-memory summarizer (best-effort, fail-soft, reuses the modelProvider SSOT).
  • The AGENT_MEMORY.miniSummary graph-node property + the query_recent_turns reader.
  • Un-summarized memories are queryable: AGENT_MEMORY graph nodes (GraphService.db.storage.db) where json_extract(memory.data, '$.properties.miniSummary') IS NULL.

The Fix

A memory-miniSummary backfill mirroring the session pattern — a lifecycle script (e.g. ai/scripts/lifecycle/backfill-memory-summaries.mjs) + an Orchestrator task definition, that:

  1. Finds AGENT_MEMORY nodes with miniSummary IS NULL, ORDER BY timestamp DESC (most-recent-first), LIMIT a batch (reuse summarizationBatchLimit or a dedicated leaf).
  2. For each, fetches prompt/response from Chroma (by node id) and calls MemoryService.buildMiniSummary(...).
  3. Updates the node's miniSummary property — best-effort/fail-soft: a failed summary leaves the node un-summarized for a later pass; never blocks the batch.
  4. Runs incrementally (batch per scheduled tick) until drained, via the Orchestrator schedule.

Contract Completeness

No new public/consumed surface: this reuses #12671's miniSummary node property, buildMiniSummary, and the modelProvider SSOT, plus the established Orchestrator task-definition mechanism. The only new artifact is an internal lifecycle script + its task definition — no MCP-tool or config-contract change beyond an optional batch-limit leaf.

Decision Record impact

aligned-with #12671 (AC8) — reuses its summarizer + node property + the modelProvider reactive-Provider SSOT (no new summarizer, no parallel provider config).

Acceptance Criteria

  • A batch finder returns AGENT_MEMORY nodes with miniSummary IS NULL, most-recent-first, limited to a batch size.
  • Each batch summarizes via buildMiniSummary and updates the node's miniSummary; fail-soft — a model failure leaves the node un-summarized for a later pass and never throws out of the batch.
  • Scheduled via the Orchestrator (mirroring summarize-sessions), running incrementally until the un-summarized set is drained.
  • Multi-tenant-safe: the backfill enriches nodes in place; the summary is tenant-agnostic enrichment (no cross-tenant content leak beyond a node's own prompt/response).
  • Test: a node with miniSummary IS NULL gets a summary after a backfill pass (stubbed model → deterministic); a stubbed model failure leaves it null (re-tryable next pass).

Out of Scope

  • The inline forward-looking summary (shipped in #12671 AC8).
  • Mini-summary AGGREGATION / temporal-pyramid (#11376 — separate Phase-2 lane).
  • Re-summarizing already-summarized memories.

Related

  • Follows #12671 (the recency primitive + the AC8 inline summary this backfills) · graduated from #12669 · mirrors the session-summarization pipeline, Related: #12065 (Orchestrator-as-SSOT for the REM/Sandman Pipeline).

Retrieval Hint: "miniSummary backfill AGENT_MEMORY most-recent-first batch mirror summarize-sessions buildMiniSummary modelProvider"

Authored by Claude (Opus 4.8, neo-opus-grace).

tobiu referenced in commit cee941a - "Add query_recent_turns — chronological recency memory-recall MCP tool (#12672) on Jun 7, 2026, 5:40 AM
tobiu referenced in commit 2599576 - "feat(memory-core): backfill memory miniSummary nodes (#12673) (#12676)" on Jun 7, 2026, 6:25 PM
tobiu closed this issue on Jun 7, 2026, 6:25 PM