Frontmatter
| id | 9094 |
| title | Feat: Batch Streaming & Progressive Rendering for Proxy Loader |
| state | Closed |
| labels | aiperformancecore |
| assignees | tobiu |
| createdAt | Feb 10, 2026, 8:16 PM |
| updatedAt | Feb 10, 2026, 8:24 PM |
| githubUrl | https://github.com/neomjs/neo/issues/9094 |
| author | tobiu |
| commentsCount | 1 |
| parentIssue | 9089 |
| subIssues | [] |
| subIssuesCompleted | 0 |
| subIssuesTotal | 0 |
| blockedBy | [] |
| blocking | [] |
| closedAt | Feb 10, 2026, 8:24 PM |
Feat: Batch Streaming & Progressive Rendering for Proxy Loader
tobiu added parent issue #9089 on Feb 10, 2026, 8:17 PM

tobiu
Feb 10, 2026, 8:18 PM
Input from Gemini 3 Pro:
✦ I have implemented the batching and progressive rendering optimizations.
Changes
src/data/proxy/Stream.mjs: AddedchunkSize: 500. Thereadloop now buffers records and fires thedataevent with an array of 500 items at a time.src/data/Store.mjs:
- Removed
suspendEventsduring the proxy load.- Updated the
onDatahandler to accept an array andadd()it directly.- Set
isLoading = falseimmediately after the first chunk arrives to unmask the grid.- Fires
loadevent after every chunk with the updatedtotalcount, allowing the Grid to update its scrollbar and row count progressively.Verification
- Updated
test/playwright/unit/data/proxy/Stream.spec.mjsto expect array payloads.- Updated
test/playwright/unit/data/StoreProxy.spec.mjsto verify progressive loading behavior.- Tests passed.
This should resolve the 10s regression and provide immediate visual feedback.
tobiu assigned to @tobiu on Feb 10, 2026, 8:22 PM
tobiu closed this issue on Feb 10, 2026, 8:24 PM
Feat: Batch Streaming & Progressive Rendering for Proxy Loader
Objective
Optimize the
Neo.data.proxy.StreamandNeo.data.Storeintegration to solve the performance regression caused by single-record event firing. Implement chunked streaming and progressive UI updates.Problem
The initial implementation fired a
dataevent for every single record in the NDJSON stream. For 11k records, this caused massive overhead (11k events, 11k store updates, 11k microtasks), blocking the App Worker for ~10s.Solution
chunkSize): UpdateStream.mjsto buffer parsed records and fire thedataevent only whenchunkSize(e.g., 500) is reached.Store.mjsto removesuspendEventsduring the stream. Instead, listen for the chunkeddataevents, add the chunk to the store, and fire aloadevent immediately. This allows the Grid to render the first chunk (~500 rows) almost instantly while the rest of the stream continues in the background.Tasks
chunkSizeconfig toNeo.data.proxy.Stream.Stream.read()to yield arrays of records.Neo.data.Store.load()to support progressive loading (removesuspendEvents, fire intermediateloadevents).Stream.spec.mjs,StoreProxy.spec.mjs).Expected Outcome