LearnNewsExamplesServices
Frontmatter
id9451
titleCreate Pipeline Cornerstone and Refactor Store Implementation
stateClosed
labels
enhancementaiarchitecturecore
assigneestobiu
createdAtMar 12, 2026, 7:22 PM
updatedAtMar 17, 2026, 7:57 PM
githubUrlhttps://github.com/neomjs/neo/issues/9451
authortobiu
commentsCount0
parentIssue9449
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtMar 17, 2026, 7:57 PM

Create Pipeline Cornerstone and Refactor Store Implementation

Closed v12.1.0 enhancementaiarchitecturecore
tobiu
tobiu commented on Mar 12, 2026, 7:22 PM

Goal

Establish Neo.data.Pipeline as the central orchestrator for data transformation and remote execution, and remove the brittle remote instantiation logic from Neo.data.Store.

Context

Currently, Neo.data.Store attempts to directly manage the cross-worker instantiation of its Normalizer (via afterSetNormalizer). This is an abstraction leak; a Store should not be hardcoded to Neo.worker.Data or manage remote IDs.

We need a dedicated Neo.data.Pipeline class. The Store will aggregate a Pipeline using ClassSystemUtil.beforeSetInstance(). The Pipeline takes over the responsibility of owning the Connection, Parser, and Normalizer, and importantly, orchestrating whether they run locally in the App Worker or remotely in the Data Worker.

Acceptance Criteria

  • Create src/data/Pipeline.mjs extending Neo.core.Base.
  • Give Pipeline the following reactive configs: workerExecution (default 'app'), connection_, parser_, and normalizer_.
  • Use ClassSystemUtil.beforeSetInstance inside the Pipeline to instantiate these sub-components.
  • If workerExecution: 'data', the Pipeline should use Neo.worker.Data.createInstance to spawn the actual Connection, Parser, and Normalizer instances exclusively inside the Data Worker (meaning the App Worker Pipeline only holds the configs, not the instances), storing the remote ID.
  • Refactor Neo.data.Store: Remove afterSetNormalizer and afterSetParser. Introduce a pipeline_ config that uses ClassSystemUtil.beforeSetInstance to create the Pipeline.
  • The Store's load() method should delegate to this.pipeline.read().
tobiu added the enhancement label on Mar 12, 2026, 7:22 PM
tobiu added the ai label on Mar 12, 2026, 7:22 PM
tobiu added the architecture label on Mar 12, 2026, 7:22 PM
tobiu added the core label on Mar 12, 2026, 7:22 PM
tobiu added parent issue #9449 on Mar 12, 2026, 7:23 PM
tobiu cross-referenced by #9449 on Mar 12, 2026, 7:24 PM
tobiu assigned to @tobiu on Mar 12, 2026, 7:25 PM
tobiu changed title from Create Connection.Base and Establish Connection -> Parser Hierarchy to Create Pipeline Cornerstone and Refactor Store Implementation on Mar 12, 2026, 10:02 PM
tobiu cross-referenced by #9453 on Mar 12, 2026, 10:03 PM
tobiu cross-referenced by #9502 on Mar 17, 2026, 6:46 PM
tobiu referenced in commit 6c3cc3a - "feat(data): Create Pipeline Cornerstone (#9451) on Mar 17, 2026, 7:00 PM
tobiu referenced in commit e6538be - "fix(data): Refine Pipeline IPC and unbreak Store load (#9451) on Mar 17, 2026, 7:00 PM
tobiu referenced in commit 9f20ea5 - "refactor(data): Refine Pipeline IPC and improve self-healing (#9451) on Mar 17, 2026, 7:00 PM
tobiu closed this issue on Mar 17, 2026, 7:57 PM