LearnNewsExamplesServices
Frontmatter
id14640
titleNL tour-runner harness: scripted operation sequences with deterministic timing
stateClosed
labels
enhancementaiarchitecture
assigneesneo-fable-clio
createdAtJul 4, 2026, 5:31 AM
updatedAt12:39 AM
githubUrlhttps://github.com/neomjs/neo/issues/14640
authorneo-fable-clio
commentsCount2
parentIssue13158
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAt12:39 AM

NL tour-runner harness: scripted operation sequences with deterministic timing

Closed Backlog/active-chunk-3 enhancementaiarchitecture
neo-fable-clio
neo-fable-clio commented on Jul 4, 2026, 5:31 AM

Context

Tree line A4 of the #13158 decomposition — the trinity enabler: one scripted, deterministic NL-driven operation sequence serves as the demo (viewers watch it), the e2e scenario (#14591 replays it as a spec), and the video take (screen-record a run; no hand-driving). The operator's YouTube seed (2026-07-04) is this leaf's product justification; #14589/#14590's tour-mode ACs and #14591's replay AC all consume it.

The Problem

Without a runner, every tour is ad-hoc glue: demos hand-roll their own sequencing, e2e re-implements the same choreography with test-local timing, and recordings drift from both. Three copies of one artifact is the drift machine. The runner makes the tour script THE single source: a JSON-first scene list executed identically in demo mode, spec mode, and recording mode.

The Architectural Reality

  • Operation substrate: the #14625 tools (execute_dock_operation returning post-op documents; get_dock_topology for assertions) — the runner composes THEM, never touches the dock model directly (no parallel mutation path, the standing guardrail).
  • Determinism constraints: scene steps advance on state/event settledness, never wall-clock sleeps (the whitebox-e2e no-sleeps rule lifted into the runner) — timing configs affect PACE (inter-scene pauses for viewers), not CORRECTNESS (op ordering/settling).
  • Placement candidates (structural-pre-flight at claim): the runner is app-consumable demo infrastructure — likely apps/agentos/ module scope or an examples/ shared util; NOT ai/ (it runs App-side). Sibling sweep decides.
  • Script shape: JSON-first {scenes: [{title, caption, steps: [{op-descriptor | topology-assert | pause}]}]} — no functions, no live refs (recordable, diffable, reviewable).

The Fix

One PR: the tour-runner module + the script schema + one executable smoke script:

  1. Script schema + validator (fail-closed: unknown step types rejected with the vocabulary; every op descriptor validated against the #14587 eight-op set).
  2. The runner: executes a script against a live workspace via the NL tools; three modes — demo (paced, captions surfaced), spec (no pacing, assertions strict), record (paced, deterministic seed, reduced-motion OFF asserted).
  3. Settledness contract: each op step completes on the returned post-op document matching an expected topology predicate (or an explicit topology-assert step) — never setTimeout correctness.
  4. Smoke script: a 3-step sequence against examples/dashboard/dock proving all three modes on shipped ops.

Acceptance Criteria

  • Script schema documented + validator fail-closed (unknown steps/ops enumerated).
  • Runner executes the same script byte-identically in demo/spec/record modes (mode changes pace/output, never order).
  • Zero wall-clock correctness: settledness via post-op documents/topology asserts only (review-checkable).
  • Smoke script green against examples/dashboard/dock; two consecutive runs produce identical operation logs (the determinism falsifier).
  • Unit specs for schema validation + mode equivalence (custom playwright config).
  • Cross-family review before merge.

Out of Scope

  • The demo scenes themselves (F3–F5, G2–G4 — scripts are content, this is the player).
  • Recording tooling/ffmpeg (F7).
  • Perspective/cross-window steps until their tools exist (A1 — the schema reserves the step types, marked unavailable-fail-closed).

Related

Parent: #13158 (tree line A4) · consumes #14587/#14625 tools · consumed by #14589/#14590 tour modes + #14591 replay spec · sibling A3 (topology diff tool — the assert step's future upgrade).

Live latest-open sweep: latest 10 checked 2026-07-04T03:31Z; no equivalent. A2A in-flight: map line unclaimed; no collision.

Origin Session ID: fa2a6fd5-7488-4af6-a0d2-3855c86003e4 Retrieval Hint: "NL tour runner scripted scenes demo spec record modes settledness"

Contract Ledger Matrix (backfilled per PR #14912 review cycles 1–2)

The consolidated matrix for the script format, validator exports, runner API, log wire format and workspace advisory lives in this ticket comment and is normative for the shipped surfaces; this body section is the required in-body anchor.

Evidence Residual (evidence-ladder disposition)

  • Open AC: "smoke script green against examples/dashboard/dock" — the named LIVE page replay. Not delivered by PR #14912 (which delivers L2 fixture-parity on the identical document topology + a live dockdemo childapp exercising the same seam).
  • Disposition: L3 delivery rides #14591 (the whitebox-e2e leaf that consumes the runner + screenplay). This ticket stays OPEN until that live smoke lands; PR #14912 references it as Refs, not Resolves.
tobiu referenced in commit e2d39c2 - "feat(ai): deterministic tour-runner + Demo-A dock-choreography screenplay (#14640) (#14912) on Jul 10, 2026, 7:48 AM
tobiu referenced in commit 9eafa09 - "test(dashboard): TourRunner L3 smoke — two-run log-identity replay on the dock example (#14640) (#15107) on 12:39 AM
tobiu closed this issue on 12:39 AM