Context
Born from live friction during tonight's devindex grid-corruption debugging (#12930 / #12929 family). The operator's direct challenge: "do we need to enhance NL, to make it more 'visual' for models?" This ticket is the evidence-grounded answer: for the defect class we just hit, models don't need pixels — they need rendered-geometry truth at the granularity where the defect lives, and NL is exactly one parameter-type short of providing it.
Release classification: boardless — agent-tooling enhancement, post-release.
The Problem
observe_motion samples client rects over a time window — the designated render-layer instrument (operational handbook §4.1 pairs it with get_drag_trace's logic layer). But it accepts componentIds only. Grid cells are pooled raw vdom nodes, NOT components (verify_component_consistency on any grid row shows items: 0) — so cell motion is structurally invisible to NL's motion forensics.
Tonight's two operator repros (#12930 comment IC_kwDODSospM8AAAABF0-9FA) live precisely in that blind spot:
- Repro A: drag reversal → header (a component — observable) returns, cells (nodes — invisible) don't.
- Repro B: overdrag scroll → no cells animate while header switching proceeds.
An agent with node-granularity rect sampling would have seen both DIRECTLY in one call: cell rects diverging from their header's rect (A), cell rects frozen during overdrag (B). Instead the agent (me) had to infer the defect from structural state (the id-generation doubling) and rely on the operator's eyes for the motion symptom. The handbook's own architecture is right; the instrument's reach is one level too shallow.
The Architectural Reality
observe_motion resolves componentIds → DOM ids and samples getBoundingClientRect per tick on the main thread. getBoundingClientRect is post-transform rendered geometry — for layout/transform/animation defects it IS the visual truth; pixels add nothing for this class.- Cells already have stable DOM ids (
<rowId>__cell-N / <rowId>__<field> — both schemes, per #12930). The sampler needs only to accept raw element ids; the id→element resolution is the entire delta.
- The genuinely-pixel tier (paint-level defects: z-order, clipping, opacity, GPU-layer artifacts — where geometry can lie) stays out of scope: that's screenshot territory, served today by browser-MCP/Playwright outside NL. Keeping NL geometry-true and delegating paint-truth is the right boundary.
The Fix
observe_motion accepts nodeIds: string[] (raw DOM element ids) alongside componentIds — same sampler, same trace shape, per-id resolution fallback Neo.getComponent(id) || document.getElementById(id) on the main-thread side.- Optional convenience:
cellsOf: {rowId} expanding to that row's current cell nodes (saves the agent a consistency-call round trip).
- Doc delta: handbook §4.1 gains the node-granularity guidance + the "geometry-truth vs paint-truth" boundary note.
- (Follow-up candidate, separate ticket if wanted: declarative motion assertions for
/whitebox-e2e — "these node rects must return to baseline after reversal" — turning tonight's repro A into a permanent regression test.)
Contract Ledger Matrix
(added 2026-06-12 per peer intake feedback — the tool-surface change needs row-level anchors before implementation)
| Target Surface |
Source of Authority |
Proposed Behavior |
Fallback |
Docs |
Evidence |
observe_motion MCP tool schema (EXISTING, modified) |
ai/mcp/server/neural-link/openapi.yaml:656-678 (/component/observeMotion, required: [componentIds]) |
required relaxes to "at least one of componentIds / nodeIds"; new nodeIds: string[] property (raw DOM element ids); optional cellsOf: {rowId} convenience expansion |
Calls passing only componentIds behave exactly as today (additive change) |
Tool description block in the same yaml |
Schema read 2026-06-12; x-pass-as-object: true confirms object-shaped args |
| Tool routing (EXISTING, unchanged) |
ai/mcp/server/neural-link/toolService.mjs:48 |
No change — same binding to InteractionService.observeMotion |
n/a |
n/a |
Routing line verified |
| Service relay (EXISTING, modified) |
ai/services/neural-link/InteractionService.mjs:66 (observeMotion({componentIds, durationMs, intervalMs, sessionId})) |
Adds nodeIds / cellsOf pass-through to the App-worker client |
Absent params pass undefined — current callers unaffected |
JSDoc on the method |
Signature verified |
| Runtime sampler (EXISTING, modified) |
src/ai/client/ComponentService.mjs:75-87 — validates componentIds non-empty, resolves Neo.getComponent(componentIds[0]) (component-only: THE gap), clamps durationMs 100–8000 / intervalMs 16–1000 |
Validation becomes "componentIds or nodeIds non-empty"; per-id resolution falls back component-registry → raw DOM id; trace shape unchanged ({componentIds, samples: [{t, rects}]} gains the node ids in its rect keys); clamps unchanged |
Component-only calls hit the identical code path |
Method JSDoc |
Implementation read 2026-06-12; the empty-array throw at :76-78 is the surface the new validation replaces |
| Operational handbook §4.1 (EXISTING, modified) |
.agents/skills/neural-link/references/operational-handbook.md |
Gains node-granularity guidance + the geometry-truth vs paint-truth boundary note |
n/a |
(is the docs row) |
Handbook section confirmed as the instrument-pairing source |
Evolution note (2026-06-12): Contract Ledger added in place by the ticket author after peer intake review yielded the ticket citing its absence; the Fix section's "main-thread side" resolution phrasing is superseded by the sampler row above (resolution lives in the App-worker client, raw-id rect reads relay to Main) — the implementing PR settles internals within that contract.
Acceptance Criteria
Out of Scope
- Screenshot/paint-truth capture inside NL (browser-MCP territory; revisit only if a geometry-lies defect class materializes).
- The motion-assertion DSL for whitebox-e2e (follow-up, separate scope).
- The #12930 fix itself (this instrument is its verification dependency, not its solution).
Related
- #12930 (primary consumer — bisect verification + Repro-A falsifier), #12929, #12883 (the defect family that exposed the gap), #12884 (sibling NL gap, different surface).
- Operational handbook:
.agents/skills/neural-link/references/operational-handbook.md §4.1.
Live latest-open sweep: checked latest 15 open issues at 2026-06-11T23:49Z; no equivalent found (#12884 = window-registration gap, different NL surface). A2A in-flight sweep: clean (this turn's traffic only).
Lane note: filed unassigned — it's claimable by anyone; I'll claim it when #12930's bisect reaches verification stage if it's still open, since Repro A's falsifier needs it.
Origin Session ID: e6b095bb-4d88-4aeb-a0ec-7d47e0f8f7ce
Retrieval Hint: "observe_motion node granularity cells invisible motion forensics neural link visual"
Context
Born from live friction during tonight's devindex grid-corruption debugging (#12930 / #12929 family). The operator's direct challenge: "do we need to enhance NL, to make it more 'visual' for models?" This ticket is the evidence-grounded answer: for the defect class we just hit, models don't need pixels — they need rendered-geometry truth at the granularity where the defect lives, and NL is exactly one parameter-type short of providing it.
Release classification:boardless — agent-tooling enhancement, post-release.The Problem
observe_motionsamples client rects over a time window — the designated render-layer instrument (operational handbook §4.1 pairs it withget_drag_trace's logic layer). But it acceptscomponentIdsonly. Grid cells are pooled raw vdom nodes, NOT components (verify_component_consistencyon any grid row showsitems: 0) — so cell motion is structurally invisible to NL's motion forensics.Tonight's two operator repros (#12930 comment
IC_kwDODSospM8AAAABF0-9FA) live precisely in that blind spot:An agent with node-granularity rect sampling would have seen both DIRECTLY in one call: cell rects diverging from their header's rect (A), cell rects frozen during overdrag (B). Instead the agent (me) had to infer the defect from structural state (the id-generation doubling) and rely on the operator's eyes for the motion symptom. The handbook's own architecture is right; the instrument's reach is one level too shallow.
The Architectural Reality
observe_motionresolves componentIds → DOM ids and samplesgetBoundingClientRectper tick on the main thread.getBoundingClientRectis post-transform rendered geometry — for layout/transform/animation defects it IS the visual truth; pixels add nothing for this class.<rowId>__cell-N/<rowId>__<field>— both schemes, per #12930). The sampler needs only to accept raw element ids; the id→element resolution is the entire delta.The Fix
observe_motionacceptsnodeIds: string[](raw DOM element ids) alongsidecomponentIds— same sampler, same trace shape, per-id resolution fallbackNeo.getComponent(id) || document.getElementById(id)on the main-thread side.cellsOf: {rowId}expanding to that row's current cell nodes (saves the agent a consistency-call round trip)./whitebox-e2e— "these node rects must return to baseline after reversal" — turning tonight's repro A into a permanent regression test.)Contract Ledger Matrix
(added 2026-06-12 per peer intake feedback — the tool-surface change needs row-level anchors before implementation)
observe_motionMCP tool schema (EXISTING, modified)ai/mcp/server/neural-link/openapi.yaml:656-678(/component/observeMotion,required: [componentIds])requiredrelaxes to "at least one ofcomponentIds/nodeIds"; newnodeIds: string[]property (raw DOM element ids); optionalcellsOf: {rowId}convenience expansioncomponentIdsbehave exactly as today (additive change)descriptionblock in the same yamlx-pass-as-object: trueconfirms object-shaped argsai/mcp/server/neural-link/toolService.mjs:48InteractionService.observeMotionai/services/neural-link/InteractionService.mjs:66(observeMotion({componentIds, durationMs, intervalMs, sessionId}))nodeIds/cellsOfpass-through to the App-worker clientundefined— current callers unaffectedsrc/ai/client/ComponentService.mjs:75-87— validatescomponentIdsnon-empty, resolvesNeo.getComponent(componentIds[0])(component-only: THE gap), clampsdurationMs100–8000 /intervalMs16–1000{componentIds, samples: [{t, rects}]}gains the node ids in its rect keys); clamps unchanged:76-78is the surface the new validation replaces.agents/skills/neural-link/references/operational-handbook.mdEvolution note (2026-06-12): Contract Ledger added in place by the ticket author after peer intake review yielded the ticket citing its absence; the Fix section's "main-thread side" resolution phrasing is superseded by the sampler row above (resolution lives in the App-worker client, raw-id rect reads relay to Main) — the implementing PR settles internals within that contract.
Acceptance Criteria
observe_motion({nodeIds: [...]})returns rect traces for non-component vdom nodes (grid cells as the test case).Out of Scope
Related
.agents/skills/neural-link/references/operational-handbook.md§4.1.Live latest-open sweep: checked latest 15 open issues at 2026-06-11T23:49Z; no equivalent found (#12884 = window-registration gap, different NL surface). A2A in-flight sweep: clean (this turn's traffic only).
Lane note: filed unassigned — it's claimable by anyone; I'll claim it when #12930's bisect reaches verification stage if it's still open, since Repro A's falsifier needs it.
Origin Session ID: e6b095bb-4d88-4aeb-a0ec-7d47e0f8f7ce
Retrieval Hint: "observe_motion node granularity cells invisible motion forensics neural link visual"