LearnNewsExamplesServices
Frontmatter
id16033
titleE2 leaf: the harness product witness cannot observe success — liveness selectors are pinned to the sample state
stateClosed
labels
ai
assigneesneo-opus-vega
createdAtJul 27, 2026, 12:52 AM
updatedAtJul 27, 2026, 8:36 AM
githubUrlhttps://github.com/neomjs/neo/issues/16033
authorneo-opus-vega
commentsCount1
parentIssue13377
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJul 27, 2026, 8:36 AM
milestonev13.2

E2 leaf: the harness product witness cannot observe success — liveness selectors are pinned to the sample state

neo-opus-vega
neo-opus-vega commented on Jul 27, 2026, 12:52 AM

Context

v13.2's release gate opens with "a developer downloads and runs the local harness without hand-editing config", and cornerstone 1's done-signal is "@tobiu starts an agent from the UI, not a terminal" (ROADMAP.md). The harness smoke is the only automated witness for that clause. It currently cannot report success.

Filed as an ADR-0034 §5 E2 leaf (E2 owns harness-smoke re-run ownership); steward @neo-opus-vega per #13377. Measured live on dev@b67e208115 by running npm run smoke and npm run smoke:brain in harness/.

The Problem

1. The liveness labels can only ever report the failure case. harness/preload.cjs:57,60:

activityLabel = document.querySelector('.fm-fleet-cockpit .fm-stream-head.is-sample .fm-stream-state')?.textContent?.trim() ?? null,
rosterLabel   = document.querySelector('.fm-fleet-cockpit .fm-fleet-head.is-sample .fm-fleet-stale')?.textContent?.trim() ?? null;

Both selectors are scoped to .is-sample. FleetGrid.mjs:306 sets is-${adapterState}, so a cockpit that promotes to live makes these selectors match nothing and the fields report null — indistinguishable from "selector broken" or "cockpit absent". The witness is wired exclusively to the state we do not want to ship.

2. productWitnessPassed is unreachable from the smoke scripts. harness/main.mjs:1241:

productWitnessPassed: packagedMode && brain.mode && brain.up === true && firstPaintPassed

package.json exposes smoke and smoke:brain, both electron main.mjs — never a packaged run. So packagedMode is always false and the field is always false. Measured smoke:brain on this host: brain.mode true, brain.up true, chromaListening true, firstPaint.passed true, sharedHeapEvidence true, popupMaterialized true — and productWitnessPassed false on the packagedMode conjunct alone.

Why this matters more than a test tidy-up: a field named productWitnessPassed reading false on a fully healthy boot invites exactly the wrong conclusion. I built a story about the cockpit lying to strangers before reading the definition. A witness that always fails is worse than no witness — it trains readers to discount it, and it cannot catch the regression it exists for.

The Architectural Reality

  • FleetGrid.mjs:110–113adapterState_ defaults 'live'; :306 renders is-${adapterState} on .fm-fleet-head.
  • FleetCockpit.mjs:293,427gridAdapterState / streamAdapterState seed 'sample' and promote on a capability answer (:2010 wired → live, :2015 degraded, :2028 absent → keep the sample seed).
  • So is-sample, is-live, is-stale, is-degraded are all reachable render states, and the witness reads exactly one.
  • Observed under smoke:brain with the Brain fully up: the cockpit still rendered is-sample (static roster · offline / sample · live feed pending). Whether that is correct for a checkout-isolated smoke profile with no live fleet residents, or a genuine wiring gap, is not decidable with the current instrument — which is the point of this leaf. Fix the witness first, then the answer is a measurement rather than an argument.

The Fix

  1. Read the adapter state unscoped — query .fm-fleet-head / .fm-stream-head and report the observed is-* state plus its label text, so live is a positive observation and a missing element is distinguishable from a live one.
  2. Report the state as data (rosterState: 'sample'|'live'|'stale'|'degraded'|null) alongside the label, rather than encoding the verdict in selector scope.
  3. Make the product witness honest about what it measured: either report packagedMode as its own gate so productWitnessPassed is not silently unreachable from npm run smoke, or add the packaged-run script the field presumes and name which script satisfies it.
  4. Do not weaken the gate to make it pass — an unreachable gate and a trivially-true gate are the same defect in opposite directions.

Acceptance Criteria

  • The smoke reports the cockpit's observed adapter state as data for every reachable value (sample / live / stale / degraded), not only sample.
  • A missing cockpit element and a live cockpit produce different results; neither is reported as the other.
  • productWitnessPassed is either reachable from a named script, or is accompanied by the field that explains why it is false (packagedMode), with the presumed script named in the harness README.
  • Negative control: with the cockpit forced to live (injected adapter state), the witness reports live — proving the instrument can observe success, not just failure.
  • Negative control: with the cockpit absent, the witness reports absence distinctly from both sample and live.
  • cd harness && npm run smoke and npm run smoke:brain both still pass, with their reports showing the new fields.
  • No change to cockpit render behaviour — this leaf touches the witness only. Any cockpit wiring gap it exposes is a separate leaf.

Out of Scope

  • Whether the cockpit should be live under a checkout-isolated smoke profile — that becomes answerable once the witness works, and gets its own leaf if it is a defect.
  • The packaging/signing pipeline (E6, closed) and the update channel (E7, unfiled).
  • Cockpit adapter/capability wiring itself (#14560 surface, #13015 spine).

Avoided Traps

  • Relaxing the witness to make it green. The temptation is to drop packagedMode from the conjunction so the field reads true. That converts an always-false gate into an always-true one and loses the packaged-run claim entirely.
  • Reading the label instead of the state. The current defect exists because the verdict lives in a CSS-class selector rather than in reported data. Fixing it by adding a second scoped selector (.is-live) reproduces the same shape with more branches.

Decision Record impact

none — ADR 0034 §5's E2 row already owns harness-smoke re-run; this implements it and changes no ADR authority.

Related: #13377 (epic, steward @neo-opus-vega) · ADR 0034 §5 E2 · #14560 (cockpit surface the witness observes) · #14793 (the "download and run" UX spec this measures)

Authored by Vega (@neo-opus-vega, Claude Opus 5, Claude Code)