LearnNewsExamplesServices
Frontmatter
id9189
titleEnhance Stream Proxy with DecompressionStream Support
stateClosed
labels
enhancementaiperformance
assigneestobiu
createdAtFeb 17, 2026, 1:40 AM
updatedAtFeb 17, 2026, 1:52 AM
githubUrlhttps://github.com/neomjs/neo/issues/9189
authortobiu
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtFeb 17, 2026, 1:52 AM

Enhance Stream Proxy with DecompressionStream Support

Closed v12.0.0 enhancementaiperformance
tobiu
tobiu commented on Feb 17, 2026, 1:40 AM

This task enhances the Neo.data.proxy.Stream to support client-side decompression of data streams using the browser's DecompressionStream API. This is a strategic enabler for scaling datasets (e.g., from 50k to 100k+ records) by significantly reducing network transfer size (estimated 70-80% reduction for JSONL).

Objectives:

  1. Proxy Enhancement (src/data/proxy/Stream.mjs):

    • Add a compression config ('gzip', 'deflate', or null).
    • If set, pipe the fetch response body through new DecompressionStream(this.compression) before reading.
    • Ensure compatibility with existing features (Progressive Chunking, Abort).
  2. Proof of Concept (Experiment):

    • Manually compress users.jsonl to users.jsonl.gz.
    • Configure DevIndex.store.Contributors to load the compressed file with compression: 'gzip'.
    • Verify that the app loads correctly and measure the network transfer savings.
  3. Cleanup:

    • Revert the store configuration to use the uncompressed file (to maintain meaningful git diffs for data updates).
    • Keep the Proxy capability merged.

Acceptance Criteria:

  • Stream.read() successfully decompresses a .gz file when configured.
  • Loading a compressed file triggers the same progressive events as an uncompressed one.
  • Network transfer size is significantly reduced (Validation).
  • Fallback/Error handling if DecompressionStream is missing (optional but good).
tobiu added the enhancement label on Feb 17, 2026, 1:40 AM
tobiu added the ai label on Feb 17, 2026, 1:40 AM
tobiu added the performance label on Feb 17, 2026, 1:40 AM
tobiu assigned to @tobiu on Feb 17, 2026, 1:41 AM
tobiu referenced in commit beee3f7 - "feat(proxy): Add DecompressionStream support for client-side gzip/deflate handling (#9189)" on Feb 17, 2026, 1:51 AM
tobiu
tobiu Feb 17, 2026, 1:52 AM

Input from Gemini 3 Pro:

✦ I have implemented DecompressionStream support in Neo.data.proxy.Stream.

Features:

  • Config: compression: 'gzip' | 'deflate' | null.
  • Implementation: Transparently pipes the fetch stream through the browser's native DecompressionStream.
  • Impact: Enables serving .jsonl.gz files directly.
    • Proof of Concept: Validated a ~60% size reduction (14MB -> 5.8MB) for the DevIndex dataset with full functionality.

This enhancement, combined with Progressive Chunk Sizing (#9187) and Abort support (#9188), positions the data layer to handle 100k+ record datasets efficiently.

tobiu closed this issue on Feb 17, 2026, 1:52 AM