LearnNewsExamplesServices
Frontmatter
id15110
titleCI: per-PR matrix omits e2e — dashboard/app PRs merge with zero e2e coverage (nightly runner boot-failing)
stateClosed
labels
testingbuild
assignees[]
createdAtJul 12, 2026, 11:47 PM
updatedAtJul 12, 2026, 11:52 PM
githubUrlhttps://github.com/neomjs/neo/issues/15110
authorneo-opus-grace
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJul 12, 2026, 11:52 PM

CI: per-PR matrix omits e2e — dashboard/app PRs merge with zero e2e coverage (nightly runner boot-failing)

Closed Backlog/active-chunk-5 testingbuild
neo-opus-grace
neo-opus-grace commented on Jul 12, 2026, 11:47 PM

Problem

The per-PR CI test matrix runs only unit + integration-unified.github/workflows/test.yml:132: suite: [integration-unified, unit]. The e2e config (test/playwright/playwright.config.e2e.mjs), which owns the dashboard/app-layer browser specs, is not in that matrix. It runs solely via the nightly e2e runner.

On PR #15062's CI (run 29191224427) that nightly runner was observed boot-failing: [nightlyE2eRunner] Running e2e.config.mjs … boot failed.

Net effect

Dashboard / app-layer PRs merge with zero per-PR e2e coverage. A DockTabOverflowNL.spec.mjs that passes locally is never executed by per-PR CI, so a regression in a floating-mount / render / align path is not caught pre-merge. This is live right now: the tab-overflow successor #15108 (dashboard projectTabsNode wiring) and the dense-workstation demo #15099 both rely on e2e journeys per-PR CI never runs.

(Also outside the per-PR matrix: the component + visual configs. The visual gap is relevant to #14618's FM-cockpit visual-regression baseline.)

Evidence

  • .github/workflows/test.yml:132suite: [integration-unified, unit] (the complete per-PR matrix).
  • .github/workflows/test.yml:95-108 — the isIntegrationRelevantPath whitelist + runIntegration/runUnit gates; no e2e branch exists.
  • test/playwright/playwright.config.e2e.mjs exists but is referenced by no per-PR workflow (grep of .github/workflows/).
  • PR #15062 run 29191224427 — nightly e2e runner boot-failure.

Fix direction (either / both)

  1. Add a scoped e2e job to the per-PR matrix, gated by the existing "Classify test scope" step so it runs only when dashboard/app files change (bounded cost — most PRs skip it). Mirror isIntegrationRelevantPath with an isE2eRelevantPath (e.g. src/dashboard/, apps/, test/playwright/e2e/).
  2. Fix the nightly e2e runner boot failure so nightly at least covers it as a backstop.

Acceptance

  • A dashboard/app-file change triggers an e2e job in per-PR CI (verified on a probe PR touching src/dashboard/).
  • A docs-only / unrelated PR does not trigger the e2e job (cost stays bounded — verified via the classify output).
  • The nightly e2e runner boots and runs e2e.config.mjs green (or the per-PR job supersedes it).

Homes under CI / test-infra (not app-layer). Related: #14781 (the e2e integration-journeys epic these journeys feed), #14618 (visual-regression baseline — the parallel visual-config gap), #15108 / #15099 (dashboard PRs currently merging without per-PR e2e).

Verified via gh run view on #15062 + .github/workflows/test.yml (this session). Authored by Grace (Claude Opus 4.8).