LearnNewsExamplesServices
Frontmatter
id14685
titleNightly whitebox-e2e run: unattended execution + red-report A2A digest
stateClosed
labels
enhancementaitesting
assigneesneo-opus-ada
createdAt6:10 AM
updatedAt6:18 PM
githubUrlhttps://github.com/neomjs/neo/issues/14685
authorneo-fable-clio
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAt6:18 PM

Nightly whitebox-e2e run: unattended execution + red-report A2A digest

Closed Backlog/active-chunk-3 enhancementaitesting
neo-fable-clio
neo-fable-clio commented on 6:10 AM

Context

Operator gap-hunt (2026-07-04). e2e lives OUTSIDE CI by design (failing-honest discipline; NEO_TEST_SKIP_CI owns exclusions) — but nothing RUNS it on a schedule. With the sprint landing dock e2e (#14591), FM module NL-proofs (#14563-class), and tour-replay specs, the suite grows while its execution stays manual: red states will sit undiscovered exactly like the middleware staleness did (3 weeks, nobody looked).

The Problem

Post-fable, the quality heartbeat must be unattended: run the whitebox-e2e configs nightly against dev, and deliver a RED-ONLY digest into the A2A mailbox (normal-priority, wake-tier compliant per #14576) naming the failing specs + first-error lines — red-as-pointer made push, not pull.

The Architectural Reality

Sibling precedent: the middleware 8h LaunchAgent (staged plist + hardened runner, neomjs/neo#12964 pattern — lockfile, clean-tree gate, structured logs, ChainAbort/finally hygiene). The A2A send rides the existing MessageService (system-sender class exists: @system identity). CRITICAL inherited rule: custom playwright configs only, never default npx playwright test.

The Fix

One PR: a scheduled runner script (LaunchAgent-staged, the #12964 hardening pattern) executing the named e2e configs → parse results → on-red: one A2A digest (failing spec list + first errors + run log path); on-green: silence (no noise). Config list declared in the script header (additive as suites land).

Acceptance Criteria

  • Unattended run executes all declared configs; lockfile + abort hygiene per the #12964 pattern.
  • Red digest arrives via A2A (normal priority, mailbox-drain class — never a wake storm); green runs send nothing.
  • Digest names specs + first-error lines + log path (actionable without archaeology).
  • Staged-not-auto-installed plist with activation docs.
  • Cross-family review.

Out of Scope

CI integration (the outside-CI discipline stands) · fixing reds (the digest points, owners fix) · unit suites (CI owns those).

Related

#14591 + #14563-class consumers · middleware scheduler precedent (neomjs/neo#12964, PR middleware-v2#17) · #14576 wake tiers · failing-honest discipline (operator-corrected precedent). Sweeps: epic+search 04:02Z, no equivalent. Origin Session ID: fa2a6fd5-7488-4af6-a0d2-3855c86003e4 Retrieval Hint: "nightly whitebox e2e runner red digest A2A"

tobiu referenced in commit 3652ad8 - "feat(ai): nightly whitebox-e2e runner + red-only A2A digest (#14685) (#14754) on 6:18 PM
tobiu closed this issue on 6:18 PM