Nightly e2e liveness reaches no reader: silence is undecidable

⚠️ Scope widened 2026-08-24 by this leaf's own AC-1 measurement (see the AC-1 comment below). The plane is not shared, and not only for the file: host-side and container-side Memory Core writes land in two different stores. The runner's RED digest therefore terminates in the same disconnected store as the liveness receipt. This leaf owns the transport for both — same channel, same writer, same reader — rather than splitting one transport across two tickets. PR #17693's wakeSuppressed: false repair stays merged: it is necessary, just not sufficient.

Context

Successor to #17691, which owned three independent silences in ai/scripts/lifecycle/nightlyE2eRunner.mjs. Defects A (the RED digest woke nobody) and C (a failed digest send lost the red) shipped in PR #17693. Defect B was withdrawn from that leaf and re-homed here, because review falsified its remedy — not its diagnosis.

The withdrawn remedy was "publish the receipt on the Memory Core healthcheck". @neo-gpt-emmy showed it had no carrier: canonical mc-server (ai/deploy/docker-compose.yml) mounts only sqlite, handoff:ro, deployment-state:ro, vector-generation and heap-observation — no nightly-e2e — so a relative path resolves inside the container's own layer and never reaches the host LaunchAgent's receipt. It would have reported unavailable permanently, from transport, which is the exact value the field exists to disambiguate.

This ticket corrects a second framing error — mine. The #17691 amendment (and my PR #17693 review response) concluded that "a host LaunchAgent writing a host checkout and a container reading its own layer have no carrier today, and designing one is an architectural question". That is true of the file. It is not true of the runner, and I did not check the runner before writing it. See The Architectural Reality.

The Problem

nightlyE2eRunner.mjs:47-49 writes .neo-ai-data/nightly-e2e/last-run.json on every run, red or green. Since PR #17693 that receipt is materially richer — it carries an explicit, recorded (never derived) delivery disposition:

{at, red, digest: 'not-required'|'pending'|'sent'|'failed', digestError?, configs, logPath}

Nothing reads it. Verified at current dev: grep -rn "nightly-e2e" across ai/, src/, test/, .github/ and learn/ returns only the plist template and the README — and the README's only "reader" is an instruction for a human to cat the file by hand. There is no programmatic consumer, no healthcheck field, no digest, no dashboard.

The runner's contract is "Green = silence." Combined with an unpublished receipt, absence of a digest means one of four things:

1. all configs green,
2. the runner never fired (it has never been installed on the canonical host),
3. it crashed before reporting,
4. it detected a red and the digest send failed.

A reader cannot tell which. PR #17693 made the receipt able to distinguish all four — digest: 'pending' vs 'failed' vs 'sent' vs 'not-required' is exactly that discrimination — and then left it on disk where no reader is looking. The measurement now exists and still reaches no decision. That is a strictly worse state to leave standing than before, because the receipt's new precision reads as though someone is consuming it.

This is not hypothetical: the layer sat at 17 red specs with total silence, indistinguishable from health, which is what #17596 exists to triage.

The Architectural Reality

The "no carrier" conclusion was drawn from the container mount table alone. The runner itself was never inspected. It is not a file-only process:

// ai/scripts/lifecycle/nightlyE2eRunner.mjs:31-34
import GraphService          from '../../services/memory-core/GraphService.mjs';
import LifecycleService      from '../../services/memory-core/lifecycle/SystemLifecycleService.mjs';
import MailboxService        from '../../services/memory-core/MailboxService.mjs';
import RequestContextService from '../../mcp/server/shared/services/RequestContextService.mjs';

The runner already holds an in-process Memory Core service connection and already performs a durable, swarm-visible write through it — await lifecycleReady(); await graphReady(); … addMessage(…). So the premise "the host and the reader have no shared channel" is false as stated: they have one, it is exercised on every red run, and defect A's repair in PR #17693 depended on it working.

Two properties bound the design, and both are structural rather than incidental:

• The service reach is inside the red branch only. nightlyE2eRunner.mjs:158-161 returns early on green (if (!red) { …; return {red: false, sent: false} }) before touching any service. A green run therefore publishes nothing anywhere — which is correct for a digest and is precisely wrong for liveness, because liveness is the fact that only a green run can establish.
• Plane identity is the open question, and it is the one to answer first. The runner writes through host-side service imports against the host plane root; production mc-server serves /app/.neo-ai-data/* from named volumes. Whether a host-written record is visible to a containerized reader is not established by the digest working — a maintainer reading their mailbox is not the same reader as mc-server. This must be measured, not assumed.

The heap-observation mount comment in ai/deploy/docker-compose.yml remains the precedent for what a real carrier looks like when one is genuinely needed, and states this ticket's whole failure class in the general case:

"Without a shared mount the reporter writes into the container's own layer and the orchestrator reads its own empty one, so every observation surfaces as unavailable/absent forever — a silent no-op, because a successful LOCAL write is indistinguishable from a delivered one at the writer."

That last clause is the invariant this ticket must satisfy however it is built: a writer must not be able to confuse a successful local write with a delivered one.

The Fix

Answer the reachability question empirically first, then publish through the cheapest channel that survives it. Do not begin with a mount.

1. Measure plane reachability before choosing a transport. Determine whether a record written by the host runner's in-process Memory Core services is readable by the canonical containerized mc-server, and record the result as a fact in this ticket. This single measurement selects between steps 2 and 3 and makes the rest of the work small.
2. If the plane is shared: publish liveness through the connection the runner already owns, and lift the service-ready calls out of the red branch so a green run also records that it ran. No mount, no new transport, no new file.
3. If the plane is not shared: add the missing half of the channel explicitly, following the heap-observation two-named-halves precedent, and name both halves in the compose comment. Only then does a mount become the right answer.
4. Publish where a reader already looks. The consuming surface must make the four states above distinguishable, reusing #17495's observed vs unavailable vocabulary rather than inventing a parallel one — a never-ran runner reports unavailable, never green.
5. Make delivery non-inferable at the writer. Whatever the transport, the publication step must be able to fail loudly. A successful local write must never be reported as a delivered one — the heap-observation invariant, applied to this channel.

Contract Ledger

Decision Record impact

none — this consumes existing healthcheck/observability vocabulary and an existing service connection. Should step 1 conclude that a new host↔container transport is required, that conclusion is itself a decision-record candidate and must be raised before implementing step 3.

Acceptance Criteria

• Plane reachability between the host runner and canonical mc-server is measured, and the result is recorded on this ticket as a fact with the command that produced it. Done — not shared; see the AC-1 comment.

• The RED digest reaches a reader the swarm actually consults, not only the store the runner can write to. A green first run must be able to confirm delivery rather than only a successful local write.

• Run liveness is published on a surface a reader already consults; the publication path is selected by that measurement, not assumed.

• The four states — all-green, never-ran, crashed-before-reporting, digest-send-failed — are distinguishable by a reader, using #17495's observed / unavailable vocabulary.

• A never-installed runner reports unavailable, never green. Absence of data is never rendered as health.

• A green run publishes liveness, emits no digest, and wakes nobody — the #17691 AC that green stays silent must not regress.

• A stale receipt reports its age rather than a verdict, and the age is the receipt's own, never restamped by the reader.

• A publication failure is observable at the writer and cannot be recorded as a delivered publication.

• Red-proof: reverting each repair independently turns its own arm red and no other.

• The digest's sender identity is chosen and stated, not inherited from whichever credential the transport authenticates. Over streamable-http the MC server binds identity from the Bearer token (Server.mjs:708), so the runner's current NEO_AGENT_IDENTITY || '@system' choice does not survive the conversion. Preserve @system and name the mechanism, or drop it deliberately and record what a reader should infer from the new sender.

• The LaunchAgent environment supplies whatever the chosen transport requires, and a missing credential fails loudly at startup rather than silently every night. The plist template declares no EnvironmentVariables key at all, while the memory-core client entry declares requiredEnv. A run that cannot authenticate must never resemble a run with nothing to report.

• A no-credential arm exists specifically, because an interactive shell exports the credential and a launchd session does not — the failure is invisible to every green local suite.

• Host-store retirement gate (cross-pinned with #16202, @neo-opus-vega). If the shared host plane store is retired before #16202 lands, nl_action_log and nl_transaction_archive are re-measured immediately before the destructive step, and a non-zero count refuses it. The two zeros measured on 2026-08-24 are not the same null: nl_action_log is gated off by actionLoggingEnabled, while nl_transaction_archive sits behind a live, ungated writer (RecorderService.mjs:349, guarded only by correctness checks). Five checkouts share that one file, so any seat calling save_transaction writes a row — empty when scoped is not empty when landed. Pinned here so the gate holds whichever of #17708 / #16202 lands first, and neither depends on the other's ordering assumption.

Out of Scope

• Activating the LaunchAgent or choosing its schedule — operator-owned, tracked on #17596.
• Fixing any of the red e2e specs — #17596 owns that triage.
• The general AGENT:* attention-set redesign — #15919 owns it.
• Re-opening the wake-tier and send-failure repairs for defects A and C — shipped in PR #17693 and correct as merged. Their transport is in scope here, which is a different contract from their wake tier.
• Adding e2e to CI, which is GPU-gated by construction and unbuildable there.

Avoided Traps

• Start by adding a nightly-e2e mount. Rejected as the opening move. It presumes the container boundary is the obstacle, which is the same unchecked assumption that produced the withdrawn remedy — in the opposite direction. Measure first; a mount may turn out to be unnecessary.
• Re-derive liveness from the digest's absence. Rejected: that is the defect. Absence is exactly what cannot be interpreted.
• Reuse the reader-side freshness pattern that restamps a cached timestamp. Rejected explicitly — PR #17693's review found #buildRequestFreshCachedHealth returning {...cachedHealth, timestamp: <now>}, which freezes ageMs while a fresh timestamp vouches for it. The receipt's own at is the only honest age source here.
• Normalize a missing or non-boolean red to green. Rejected: fail closed. The same polarity defect was caught and withdrawn on PR #17693.
• Treat "the digest reaches the mailbox" as proof the plane is shared. Rejected: a maintainer reading a mailbox is a different reader from mc-server reading a mounted path. The two claims are independent and only one is established.

Related

Predecessor: #17691 (defects A + C shipped via PR #17693; this leaf owns B)
Triage of the underlying reds: #17596
Wake attention-set redesign: #15919
Observability vocabulary precedent: #17495
Carrier precedent: the heap-observation mount in ai/deploy/docker-compose.yml
Source module: ai/scripts/lifecycle/nightlyE2eRunner.mjs

Live latest-open sweep: checked latest 20 open issues at 2026-08-24T14:39Z; A2A all-status claim sweep over the last 30 messages at 2026-08-24T14:41Z; no equivalent ticket and no competing claim. Memory Core semantic sweep was a clear miss — live source and the compose mount table decide.

Origin Session ID: 728a756d-71df-48e6-8dad-0bac498ca23e

Retrieval Hint: query_raw_memories("nightly e2e liveness receipt reaches no reader plane reachability")
Retrieval Hint: nightlyE2eRunner last-run.json digest disposition unpublished; #17691 defect B re-homed