LearnNewsExamplesServices
Frontmatter
id8189
titleFeat: Neural Link - Advanced Event Simulation
stateClosed
labels
enhancementai
assigneestobiu
createdAtDec 28, 2025, 10:19 PM
updatedAtJan 2, 2026, 12:52 PM
githubUrlhttps://github.com/neomjs/neo/issues/8189
authortobiu
commentsCount1
parentIssue8169
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJan 2, 2026, 12:52 PM

Feat: Neural Link - Advanced Event Simulation

Closed v11.18.0 enhancementai
tobiu
tobiu commented on Dec 28, 2025, 10:19 PM

Context: Agents need to verify interactions beyond simple clicks. Simulating keyboard events and complex mouse actions is critical for E2E testing.

Scope:

  1. Create Main Thread Addon: src/main/addon/EventSimulator.mjs.

    • Expose remote methods: dispatch(nodeId, eventConfig).
    • Logic: Use new KeyboardEvent(), new MouseEvent(), etc., and element.dispatchEvent().
  2. Enhance InteractionService:

    • Add simulateEvent(sessionId, targetId, options).
    • Implementation: Call Neo.main.addon.EventSimulator.dispatch.
  3. Tools:

    • simulate_event: Generic dispatcher.

Goal: Enable comprehensive interaction testing via a new main thread addon.

tobiu added the enhancement label on Dec 28, 2025, 10:19 PM
tobiu added the ai label on Dec 28, 2025, 10:19 PM
tobiu added parent issue #8169 on Dec 28, 2025, 10:19 PM
tobiu assigned to @tobiu on Jan 2, 2026, 11:20 AM
tobiu referenced in commit 097fee8 - "Feat: Neural Link - Advanced Event Simulation #8189" on Jan 2, 2026, 12:44 PM
tobiu
tobiu Jan 2, 2026, 12:44 PM

Input from Gemini 3:

✦ I have completed the implementation of Advanced Event Simulation.

Changes from Original Plan

The final implementation diverged from the initial scope to better support cross-window orchestration and simplify the API surface:

  1. Orchestration Layer Shift:

    • Original: EventSimulator (Main Thread) was going to handle sequences.
    • Final: InteractionService (App Worker) handles the sequencing. This is critical because a Main Thread addon is bound to a single window context. By moving the loop to the App Worker, we can dispatch individual events in a sequence to different windows, enabling cross-window automation (e.g., drag-and-drop across apps).
  2. API Simplification:

    • Original: Ambiguous mix of single-event params vs. events array.
    • Final: The simulate_event tool strictly requires an events array. This provides a consistent schema for LLMs and avoids brittle oneOf logic.
  3. Explicit Routing:

    • windowId is now a mandatory string for every event in the sequence. This ensures deterministic routing in multi-window shared-worker environments.
  4. Core Enhancement:

    • Enhanced DomAccess.getElement() to natively support window, document, and document.body keywords, providing a single source of truth for target resolution.

Artifacts

  • src/main/addon/EventSimulator.mjs: Native event dispatcher.
  • src/ai/client/InteractionService.mjs: Orchestrator service.
  • ai/mcp/server/neural-link/openapi.yaml: Schema definition.
  • src/main/DomAccess.mjs: Core enhancement.
tobiu referenced in commit e34a877 - "#8189 yaml file fix" on Jan 2, 2026, 12:50 PM
tobiu closed this issue on Jan 2, 2026, 12:52 PM