Per-turn mini-summaries: short-term-recall substrate for "what just happened" queries

Author's Note: Filed by Claude Opus 4.7 (Claude Code) during session b5a17132-7324-46e1-b73e-038825bb4d55 after @tobiu surfaced the gap this turn. "query_memories is great for semantic search, but not for a quick history check" — empirically true. The substrate currently has no path for "show me the last N turns chronologically per agent" without expensive scans. Mini-summaries close the gap.

2026-04-25 author correction by @tobiu: my original v1 recommendation ("property on existing MEMORY node, use Nodes.data JSON column") was architecturally invalid — Nodes is the graph (better-sqlite3) substrate; AgentIdentity / MESSAGE / CONCEPT entities live there. Memories live in Chroma only (neo-agent-memory collection, 8765 entries). There ARE no memory rows in the graph DB. I conflated graph and memory storage. Body below corrected; placement re-opened as exploratory between Chroma-metadata-extension vs new-table-in-graph-DB. Per @tobiu: "this topic might need a vague ticket or ideation sandbox, to properly refine what makes sense, and tackle it inside a fresh session." Treating this ticket as an exploratory placeholder — detailed scope/architecture deferred to a fresh session for proper refinement, possibly graduating to a Discussion in ideation-sandbox if the OQ surface widens.

Context

Each add_memory call this session-arc has produced 2-10KB of substance (prompt + thought + response). Healthcheck shows 8765 memories in Chroma neo-agent-memory collection. For history-scan queries — "what did Claude do in the last 20 turns?" — current paths are:

• query_raw_memories: Chroma vector search; fast for relevance but doesn't preserve chronology
• query_summaries: session-grain (794 entries in neo-agent-sessions), too coarse for turn-grain recall
• Direct Chroma scan with metadata filter: would work but expensive at scale

The "amnesia returning" pattern @tobiu flagged this turn — re-derivations across sessions, missed Sandman handoffs at context-prune resume — shows the short-term-recall faculty has no fast path. Semantic retrieval finds related memories; chronological retrieval surfaces recent memories. Both load-bearing, currently only the first is solved.

Proposal

Generate a ~256-char-ish (or token-budget-equivalent) mini-summary at each add_memory call (or async post-write) using either Gemini Flash (remote, cheap, fast) or Gemma4-31B (local, offline). Surface via a new MCP tool e.g. query_recent_turns({agentId, limit}) returning chronologically-ordered mini-summaries scoped to one agent.

Open Questions for Fresh-Session Refinement

This is the architectural decision surface — not scope-frozen. To be refined in a fresh session, possibly via ideation-sandbox if OQ count grows.

OQ 1: Storage placement — Chroma metadata vs new SQL table in graph DB

Option A — Chroma metadata extension on existing neo-agent-memory collection:

• Each memory's metadata already has agent, sessionId, timestamp, model, etc.
• Add miniSummary field to metadata
• Query path: Chroma metadata filter (where: {agent: '@neo-opus-ada'}) sorted by timestamp + LIMIT N
• Pros: single source of truth (memory + summary co-located); no new schema; uses existing Chroma collection
• Cons: Chroma is optimized for vector search, not chronological scans; metadata-only queries may be slower than indexed SQL; pollutes embedding-substrate concerns with non-vector lookup pattern

Option B — New table in existing better-sqlite3 graph DB (NOT a new database):

• New MemoryMiniSummaries table with (agentId TEXT, memoryId TEXT, createdAt TEXT, summary TEXT), indexed on (agentId, createdAt DESC)
• Cross-references the Chroma memory by ID
• Pros: indexed SQL is idiomatic for chronological queries; ~10ms vs ~100ms for last-20 scan; doesn't pollute Chroma's vector-search substrate
• Cons: new schema; sync risk between Chroma memory write and SQL summary write; cross-substrate consistency model needed

Option C — New separate SQLite DB just for mini-summaries: Rejected per @tobiu — extension over fragmentation; if SQL is the choice, use existing graph DB with new table.

Recommendation deferred to fresh-session refinement. Both A and B are valid; the trade-off is "single-source-of-truth in Chroma" vs "indexed-SQL chronological-scan performance." Empirical benchmark + use-case-frequency analysis would inform the right choice.

OQ 2: Summary length anchor

@tobiu invited the challenge. Per feedback_challenge_numerical_thresholds_in_acs memory, hardcoded char limits trigger Hypothesis-Gate scrutiny. Alternatives:

• Token budget (e.g. 64 tokens ≈ 256 chars in English) — substrate-agnostic; aligns with how LLMs count
• Adaptive — compress to N% of original size with floor/ceiling
• Configurable per agent — Gemini's turns may need different summary length than Claude's

Recommendation: start with 64-token budget + log actual lengths + revisit if distribution is bimodal.

OQ 3: Synchronization timing — sync at add_memory vs async daemon

If summarization fires synchronously, every memory write blocks on a Flash/Gemma4 round-trip (200ms-2s).

Alternatives:

• Async daemon: mini-summary daemon polls for memories lacking miniSummary, generates batched, backfills
• Inline with deferred write: add_memory writes canonical immediately, kicks off async summarization that updates when ready
• Batch at session-boundary: generate mini-summaries when a session closes

Recommendation: async daemon (similar to MemorySessionIngestor pattern in ai/daemons/services/). Memory writes stay sub-second; freshly-written memories without summaries filtered (WHERE miniSummary IS NOT NULL).

OQ 4: Model choice — Flash vs Gemma4

• Flash: remote API, 200-500ms, low per-call cost, requires network + API key
• Gemma4-31B: local Ollama, 1-3s on M-series, zero per-call cost, fully offline

Recommendation: Flash default + Gemma4 opt-in via config (mirror existing dual-model patterns).

OQ 5: Cross-agent visibility

Should one agent's mini-summaries be readable by other agents? @tobiu's framing — "another way to quickly check what happened inside the last turns" — implicitly yes for cross-agent coordination context.

If memories get a sharedEntity: true semantic flag (per #10325 graph primitive — though that's graph-only), cross-agent visibility works structurally. Caveat: memories may contain agent-private reasoning (thought field). Mini-summary should respect that — summarize prompt+response only, NOT thought, OR generate a sanitized version.

Acceptance Criteria (Exploratory)

ACs deliberately under-specified pending fresh-session refinement on the OQs above.

• OQ 1 settled: Chroma metadata vs new SQL table (with empirical benchmark if needed)
• OQ 2 settled: token-budget vs char-count anchor
• OQ 3 settled: sync vs async daemon timing
• OQ 4 settled: model selection + config
• OQ 5 settled: cross-agent visibility model + privacy-tier carve-out for thought
• New MCP tool surface: query_recent_turns({agentId, limit, since?})
• Existing query_raw_memories + query_summaries remain — complementary, not replacement
• Privacy AC: agent-private thought field NOT exposed in cross-agent-readable mini-summary
• Empirical benchmark: history-scan returns last-20-turns in target latency budget (TBD per OQ 1)

Out of Scope

• Replacing query_summaries — session-grain summaries are different scope; both substrates coexist.
• Backfilling all 8765 existing memories — daemon catches forward-looking memories; backfill is a one-shot script if/when desired (separate ticket).
• Custom summarization prompt-engineering — start with stock "summarize this turn in N tokens" prompt; iterate empirically post-v1.
• Daemon hot-reload — async daemon runs on cron/interval; runtime tuning is cleanup work.
• Conflating memories with graph nodes — I made this error in the original ticket body. Memories are Chroma-only; the graph holds AgentIdentity/MESSAGE/CONCEPT relational entities. The two substrates are distinct.

Avoided Traps

• Hardcoding 256-char limit without empirical anchor — rejected per feedback_challenge_numerical_thresholds_in_acs. Use token budget + measure.
• Synchronous summarization in add_memory — rejected; latency cost compounds at swarm scale.
• Storing mini-summary in graph as a node with relationships — rejected per @tobiu's correction; mini-summary is a property of a memory, and memories aren't graph entities.
• Property-on-graph-MEMORY-node — rejected per @tobiu's correction this turn; there ARE no memory nodes in the graph. My original recommendation was structurally invalid.
• A new separate SQLite DB just for mini-summaries — rejected per @tobiu; extension over fragmentation. If SQL, use new table in existing graph DB.
• Treating mini-summary as replacement for query_raw_memories — rejected; complementary substrates serving different query patterns.
• Auto-publishing all turn summaries to A2A mailbox — rejected; mini-summary is a passive lookup substrate, not an active push channel.

Related

• #10311 — Epic: Institutionalizing Swarm Autonomy. Mini-summaries are short-term-recall substrate; complementary to A2A wakeup events (long-term coordination) and DreamService (autonomic consolidation).
• #10309 — Discussion: substrate-query-first boot. Boot-time orientation could optionally include "last 5 mini-summaries from this agent" as one signal among many.
• #10325 — sharedEntity:true primitive in graph DB. Adjacent but distinct — graph layer; this ticket is Chroma+possibly-SQL layer.
• #10330 — single-canonical identity format migration (PR #10331 in flight). Per-agent filtering depends on identity normalization.
• ai/daemons/services/MemorySessionIngestor.mjs — substrate precedent for the proposed async daemon pattern.

Origin Session ID: b5a17132-7324-46e1-b73e-038825bb4d55 Retrieval Hint: "per-turn mini-summary short-term-recall chronological-scan agent-history Flash Gemma4 64-token Chroma-metadata-vs-SQL-table async-daemon healing-amnesia organism deferred-fresh-session"