LearnNewsExamplesServices
Frontmatter
id9420
titleMigrate Data Pipeline to Connection -> Parser -> Normalizer flow
stateClosed
labels
enhancementairefactoringarchitecturecore
assigneestobiu
createdAtMar 9, 2026, 4:46 PM
updatedAtMar 12, 2026, 4:37 PM
githubUrlhttps://github.com/neomjs/neo/issues/9420
authortobiu
commentsCount1
parentIssue9449
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtMar 12, 2026, 4:37 PM

Migrate Data Pipeline to Connection -> Parser -> Normalizer flow

Closed v12.1.0 enhancementairefactoringarchitecturecore
tobiu
tobiu commented on Mar 9, 2026, 4:46 PM

Goal

Wire the new Parser/Normalizer architecture through the entire data pipeline, ensuring the Data Worker performs all transformations before sending data back to the App Worker.

Context

With Normalizers defined and the Data Worker capable of dynamically loading them via RMA, the execution flow must be updated to shift heavy lifting out of the App Worker.

Acceptance Criteria

  • Deprecate/Rename Neo.data.proxy.* namespace in favor of Neo.data.parser.* (e.g., Neo.data.parser.Stream).
  • Store.load() calls should pass the parser and normalizer module paths down to the Connection via the worker bridge.
  • Neo.data.connection.Fetch and Neo.data.connection.Xhr (running in the Data Worker) must intercept the response, instantiate the requested Parser, pass the JS object to the requested Normalizer, and postMessage the final flattened data back to the App Worker.
tobiu added the enhancement label on Mar 9, 2026, 4:46 PM
tobiu added the ai label on Mar 9, 2026, 4:46 PM
tobiu added the refactoring label on Mar 9, 2026, 4:46 PM
tobiu added the architecture label on Mar 9, 2026, 4:46 PM
tobiu added the core label on Mar 9, 2026, 4:46 PM
tobiu added parent issue #9404 on Mar 9, 2026, 4:46 PM
tobiu assigned to @tobiu on Mar 9, 2026, 4:47 PM
tobiu cross-referenced by #9449 on Mar 12, 2026, 3:20 PM
tobiu removed parent issue #9404 on Mar 12, 2026, 3:21 PM
tobiu added parent issue #9449 on Mar 12, 2026, 3:21 PM
tobiu referenced in commit 5261e90 - "feat: Migrate Data Pipeline to Connection -> Parser -> Normalizer flow (#9420) on Mar 12, 2026, 4:36 PM
tobiu
tobiu Mar 12, 2026, 4:37 PM

Input from Gemini 3.1 Pro:

✦ I have completed the migration of the Data Pipeline to the Connection -> Parser -> Normalizer flow.

Changes Made:

  • Renamed Proxy to Parser: Moved src/data/proxy/ to src/data/parser/.
  • Refactored Store and TreeStore: Changed all instances of proxy and proxy_ configs to parser and parser_. The .load() flow now routes through the parser.
  • Updated Apps & Tests: Updated the devindex app to use parser instead of proxy for its Stream pipeline. Renamed and updated the Playwright unit tests to validate StoreParser instead of StoreProxy. All 331 unit tests continue to pass.

Committed and pushed under #9420.

tobiu closed this issue on Mar 12, 2026, 4:37 PM