LearnNewsExamplesServices
Frontmatter
id9544
titleEnhance RemoteMethodAccess to Support Instance-to-Instance Routing
stateClosed
labels
enhancementaiarchitecturecore
assigneestobiu
createdAtMar 24, 2026, 8:31 PM
updatedAtMar 25, 2026, 9:38 AM
githubUrlhttps://github.com/neomjs/neo/issues/9544
authortobiu
commentsCount1
parentIssue9449
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtMar 24, 2026, 8:44 PM

Enhance RemoteMethodAccess to Support Instance-to-Instance Routing

Closed v12.1.0 enhancementaiarchitecturecore
tobiu
tobiu commented on Mar 24, 2026, 8:31 PM

Goal

Upgrade the RemoteMethodAccess (RMA) mixin to support routing messages to specific class instances across worker boundaries, rather than being limited to static Singletons.

Context

Currently, the RMA mixin (src/worker/mixin/RemoteMethodAccess.mjs) resolves remote method targets exclusively via static namespaces (e.g., let pkg = Neo.ns(msg.remoteClassName)). This works perfectly for Singletons like Managers or Main Thread Addons.

However, to support the new Neo.data.Pipeline architecture (Epic #9449) where an App Worker Pipeline dynamically spawns and communicates with a counterpart Pipeline instance in the Data Worker, we need a generic way to target instances by their id.

Instead of hardcoding domain-specific routers (like an onPipelineExecute handler) inside Neo.worker.Data, we should elevate RMA to be a universal, bi-directional instance-to-instance communication layer.

Acceptance Criteria

  • Modify onRemoteMethod in src/worker/mixin/RemoteMethodAccess.mjs to check the incoming msg payload for a target instance ID (e.g., msg.instanceId).
  • If instanceId is provided, resolve the target object using Neo.manager.Instance.get(msg.instanceId).
  • If no ID is provided, gracefully fall back to the existing static namespace resolution (Neo.ns(className)) to maintain backward compatibility for all existing Singletons.
  • Ensure the existing robust features of RMA (Promise tracking, error catching, and zero-copy transferable handling) work seamlessly for instance-based execution.
tobiu added the enhancement label on Mar 24, 2026, 8:31 PM
tobiu added the ai label on Mar 24, 2026, 8:31 PM
tobiu added the architecture label on Mar 24, 2026, 8:31 PM
tobiu added the core label on Mar 24, 2026, 8:31 PM
tobiu added parent issue #9449 on Mar 24, 2026, 8:31 PM
tobiu referenced in commit 4e0d3ef - "feat(worker): Enhance RemoteMethodAccess to support instance-to-instance routing (#9544) on Mar 24, 2026, 8:43 PM
tobiu
tobiu Mar 24, 2026, 8:43 PM

Input from Gemini 3.1 Pro:

✦ I have successfully implemented and merged the enhancement to the RemoteMethodAccess (RMA) mixin.

Architectural Briefing: Instance-to-Instance IPC

For future developers and AI agents reviewing this architecture, it is critical to understand how RMA now bridges the isolated memory spaces of different worker threads.

The Challenge: Each worker thread (App Worker, Data Worker, etc.) has its own completely isolated Javascript memory space, its own idCounter, and its own Neo.manager.Instance registry. Because of this, an ID like pipeline-1 might exist in both the App Worker and the Data Worker simultaneously, but they refer to completely different objects in their respective memory spaces.

The Solution: The Illusion of Global IDs & The Handshake We bypass this limitation not by creating truly global IDs, but by exchanging local IDs between threads in a "Handshake":

  1. The Handshake (Exchange of Local IDs): When an instance in the App Worker (e.g., App-Pipeline, local ID: app-pipe-1) wants to spawn a counterpart in the Data Worker, it sends a createInstance IPC message. Crucially, it includes its own ID in the config payload sent over the wire: { appPipelineId: 'app-pipe-1' }. The Data Worker receives this, creates its own local instance (e.g., Data-Pipeline, local ID: data-pipe-4), and replies with its newly generated ID. Now, both instances know the local ID of their counterpart in the other thread's memory space.

  2. The Execution (Bi-Directional RMA): We enhanced src/worker/mixin/RemoteMethodAccess.mjs to recognize a new remoteId property in IPC messages.

    • App -> Data: When the App Worker wants to execute read() remotely, it calls this.generateRemote({ origin: 'data', id: 'data-pipe-4' }, 'read'). When this message arrives in the Data Worker, RMA intercepts it, sees remoteId: 'data-pipe-4', and successfully looks it up in the Data Worker's local Instance manager.
    • Data -> App: When the Data Worker needs to push streaming data back, it uses the ID it received during the handshake: this.generateRemote({ origin: 'app', id: 'app-pipe-1' }, 'onParserData'). This message arrives in the App Worker, and RMA finds it in the App Worker's local registry.

Summary By adding msg.remoteId to the RMA mixin, we have created a decoupled, bidirectional IPC layer for instances. The RMA mixin does not care how you acquired the ID; it trusts that if you provide a destination worker and a remoteId, it will route the method call directly to that instance within the target worker's isolated registry. This elegant mechanism powers the entire cross-worker Unified Data Pipeline.

tobiu closed this issue on Mar 24, 2026, 8:44 PM
tobiu referenced in commit 962da05 - "feat(worker): Enhance RemoteMethodAccess to support instance-to-instance routing (#9544) on Mar 24, 2026, 8:49 PM
tobiu referenced in commit 3c8640f - "feat(worker): Enhance RemoteMethodAccess to support instance-to-instance routing (#9544) on Mar 24, 2026, 8:51 PM
tobiu cross-referenced by #9453 on Mar 24, 2026, 9:09 PM
tobiu assigned to @tobiu on Mar 25, 2026, 9:38 AM