LearnNewsExamplesServices
Frontmatter
id8203
titleOptimizing `SummaryService.listSummaries` with Two-Phase Fetch
stateClosed
labels
enhancementairefactoringperformance
assigneestobiu
createdAtDec 29, 2025, 11:13 PM
updatedAtDec 29, 2025, 11:24 PM
githubUrlhttps://github.com/neomjs/neo/issues/8203
authortobiu
commentsCount2
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtDec 29, 2025, 11:24 PM

Optimizing SummaryService.listSummaries with Two-Phase Fetch

Closed v11.18.0 enhancementairefactoringperformance
tobiu
tobiu commented on Dec 29, 2025, 11:13 PM

ChromaDB lacks a native ORDER BY clause for metadata. The current implementation fetches all documents (limit 50), which prevents accessing older sessions and is unscalable.

Proposed Solution: Two-Phase Fetch Pattern To achieve efficient, SQL-like pagination without native DB support:

  1. Phase 1 (Lightweight): Fetch ids and metadatas for ALL sessions.
    • Note: This scales well because we avoid fetching the heavy document (text) field.
  2. In-Memory Sort: Sort the metadata array by timestamp (DESC).
  3. Pagination: Apply offset and limit to the sorted array to identify the target IDs.
  4. Phase 2 (Targeted): Fetch the full documents for ONLY the sliced IDs.

Benefits:

  • Accuracy: Guaranteed to return the true "latest" sessions.
  • Performance: Significantly reduces network payload and memory usage by only transferring necessary text data.
  • Scalability: Supports paging through the entire history, not just the random first 50.
tobiu added the enhancement label on Dec 29, 2025, 11:13 PM
tobiu added the ai label on Dec 29, 2025, 11:13 PM
tobiu added the refactoring label on Dec 29, 2025, 11:13 PM
tobiu added the performance label on Dec 29, 2025, 11:13 PM
tobiu assigned to @tobiu on Dec 29, 2025, 11:13 PM
tobiu
tobiu Dec 29, 2025, 11:14 PM

cross reference link: https://github.com/chroma-core/chroma/issues/469 => it is sad to see that the ordering feature never got implemented.

tobiu referenced in commit f562965 - "Refactor(Memory): Implement two-phase fetch for global sorting in listSummaries (#8203)" on Dec 29, 2025, 11:24 PM
tobiu
tobiu Dec 29, 2025, 11:24 PM

Implemented the two-phase fetch strategy to ensure correct global sorting and pagination of session summaries. Phase 1 fetches lightweight metadata for sorting, and Phase 2 fetches the full documents for the target slice, using a Map for O(1) re-assembly to preserve order.

tobiu closed this issue on Dec 29, 2025, 11:24 PM