LearnNewsExamplesServices
Frontmatter
id16455
titleNothing proves the update chain works end to end on a real plane
stateClosed
labels
enhancementaitestingarchitecture
assigneesneo-opus-vega
createdAtAug 3, 2026, 6:19 PM
updatedAtAug 5, 2026, 1:33 PM
githubUrlhttps://github.com/neomjs/neo/issues/16455
authorneo-opus-grace
commentsCount1
parentIssue16448
subIssues
16493 The update-chain goal bar has nowhere honest to land while it is red
subIssuesCompleted1
subIssuesTotal1
contentTrust
projected
quarantined0
signals[]
blockedBy[ ] 16320 A client''s cached MCP tool schema is pinned at connect, so a newer server capability is unreachable, [x] 16454 A lagging deployment has no plan-then-apply migration path: the config delta that prevents boot is never surfaced
blocking[]
closedAtAug 5, 2026, 1:33 PM

Nothing proves the update chain works end to end on a real plane

Closed Backlog/active-chunk-12 enhancementaitestingarchitecture
neo-opus-grace
neo-opus-grace commented on Aug 3, 2026, 6:19 PM

Refs #16448

Context

Sub of #16448, and the leaf that owns the Epic's close condition:

A deployed plane, following an allowed channel, reaches the exact eligible merged cohort without a human typing a Docker command — or fails contained, with terminal evidence its consumers can see.

This leaf exists because the Epic's first decomposition mapped every D#16304 decision onto a component and still could not update a container. Each leaf was correct; none of them owned the outcome, and nothing in graduation, family-keyed quorum or three peer reviews tested that difference. epic-create forbids an AC checklist in an Epic body — correctly, since it out-stales the linked sub graph — so the goal bar has to be a leaf whose deliverable is the proof, or it is prose nothing can bind.

Diagnosed by @neo-gpt (Euclid) as a Stage-3 decomposition failure; the dual-anchor shape (Epic outcome sentence + dedicated proof leaf) is his.

This is not #16454. That leaf wires the trigger and demonstrates the goal once, at merge time, as a recorded before/after revision pair. This leaf makes the same property standing — a scenario that fails later, when the chain rots. The distinction is load-bearing here specifically: deploy-pipeline.sh shipped a year ago, was correct the whole time, and nothing noticed that nothing called it. A one-time demonstration would not have caught that; a standing one would.

The Problem

Every existing leaf is falsifiable on its own axis, and no artifact fails when the chain does not compose.

  • #16450 proves a candidate is retained and addressable — with nothing consuming it.
  • #16451 proves selection carries a bounded hotfix obligation — with nothing selecting.
  • #16452 proves the activation kernel is the only mutation path — a negative property, satisfied perfectly by a plane that never activates at all.
  • #16453 proves admissibility is answerable — for a transition that may never be attempted.

Green on all four is compatible with a plane that never moves. That is precisely the state the Epic was filed to end, and it is the state its own decomposition could not detect. Per-leaf unit coverage measures each component against its local contract; nothing measures the goal.

This is the same class the repo has been closing elsewhere — a green surface asserting a property it does not measure — applied to a decomposition rather than to code.

The Architectural Reality

The chain the proof must exercise, end to end:

  1. a merged commit on the plane's channel;
  2. resolution to an immutable revision (git ls-remoteNEO_REVISION, PipelineWiring.md §Deployed-revision provenance);
  3. the caller notices the plane is behind and triggers (#16454);
  4. the external authority runs — ai/examples/cloud-deployment/deploy-pipeline.sh, gated by ai/scripts/maintenance/redeployPreflight.mjs;
  5. the plane reports the exact target revision at /app/.neo-revision"always populated, always true";
  6. consumers observe the transition as complete (#16320's axis);
  7. and at no point did a human type a Docker command.

The falsifier for the contained-failure limb is #16452's closure test: a durable activation receipt linking a fresh RESTORABLE result before first mutation, or no mutation — no third state.

The Fix

An executable acceptance scenario against a disposable plane, deliverable before the chain that satisfies it.

The harness lands first and lands red, because a probe written after the feature only proves the feature agreed with itself. It runs opt-in — outside the default CI gate — while the implementation leaves are open, and moves into the gate when it goes green. That flip is this leaf's completion, and it is what closes the Epic.

The scenario asserts the goal as a single property:

Given a commit merged to the plane's channel, the plane reaches that exact revision with no human-issued Docker command — or performs no mutation at all and leaves terminal evidence its consumers can read.

It supplies the cross-chain falsifier and nothing else. It implements no caller, no selector, no admissibility rule. If this leaf starts growing the machinery it measures, it has become a second umbrella wearing a sub's label, and the Epic is back to having no goal bar.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback / Error Semantics Docs Evidence
end-to-end acceptance scenario this ticket Asserts the Epic's outcome sentence as one property against a disposable plane Scenario cannot provision a plane ⇒ fails loudly as inconclusive, never as pass — an unrunnable proof must not read as a green goal scenario docs + PipelineWiring.md the scenario is red today and names which leg is missing
gate membership CI workflow Opt-in while implementation leaves are open; moves into the default gate when green Left opt-in after going green ⇒ the goal is provable and unguarded; the flip is an AC, not a follow-up workflow file the workflow runs it in the gate at close
revision assertion /app/.neo-revision Compares against the exact resolved target, not "newer than before" Missing revision file ⇒ unknown-revision ⇒ scenario fails; it never infers success from a healthy container PipelineWiring.md provenance table a plane that lands on a different commit fails the scenario

Decision Record impact

none — an acceptance harness over surfaces the sibling leaves define. It sets no contract of its own.

Acceptance Criteria

  • An executable scenario asserts the Epic's outcome sentence against a disposable deployed plane.
  • It lands red before the implementation leaves close, and its failure names the missing leg rather than failing opaquely.
  • It asserts the plane reaches the exact resolved target revision — a different commit fails, and "newer than before" is not accepted.
  • It asserts no human-issued Docker command occurred in the path under test.
  • The contained-failure limb is covered: a plane that cannot proceed performs no mutation and leaves consumer-readable terminal evidence.
  • An unrunnable scenario reports inconclusive and fails; it never reports pass.
  • It runs opt-in while siblings are open, and is moved into the default CI gate as part of this ticket once green.
  • It implements none of the machinery it measures (no caller, selector, or admissibility logic).
  • Nothing in the scenario is deployment-specific: it reads for any Neo plane on any allowed channel.

Out of Scope

  • Implementing any leg of the chain. Blocked by the implementation leaves by design.
  • Per-leaf unit coverage. Each sibling owns its own; this leaf only fails when they do not compose.
  • Performance or rollout-duration budgets. Correctness of the transition, not its speed.
  • Rollback semantics — deferred and non-gating at graduation, carried on the Epic.

Avoided Traps

  • Writing the proof after the chain. A probe authored once the feature exists tests only that the feature agrees with itself; the red-first order is what tests the probe.
  • Letting the proof leaf absorb the implementation. Named by @neo-gpt: it becomes a second umbrella disguised as a sub, and the goal bar disappears into it.
  • Asserting "newer than before". Satisfied by a plane that lands on the wrong commit; the provenance contract already distinguishes requested from resolved, and the proof must read the resolved fact.
  • Health as a proxy for arrival. A healthy container proves a container is healthy. PipelineWiring.md already bounds this: container-local reads cannot distinguish a failed rollout from a succeeded rollout whose reporter died.
  • A permanently-red test parked in the default gate. It would be routed around within a week and stop meaning anything; opt-in until green, then gated, is the honest form.

Related

  • #16448 — parent Epic; this leaf owns its close condition.
  • Blocked by #16454 (@neo-opus-vega) — the trigger, and the one-time demonstration this converts into a standing regression. Also blocked by #16450, #16451, #16452, #16453 — the remaining legs it composes.
  • #16320 — the consumer-visible completion axis this asserts against.
  • D#15758 — the authority whose apply transaction step 4 exercises.
  • learn/agentos/cloud-deployment/PipelineWiring.md — the provenance contract supplying the exact-revision assertion.

Origin Session ID: 9f05cd72-5457-4ec2-926c-ef1406041f19

Retrieval Hint: query_raw_memories("end-to-end acceptance proof leaf deployed plane exact revision no human docker command red-first goal bar")

tobiu referenced in commit 3cc89ab - "The update-chain goal bar lands red, naming which legs are missing (#16494) on Aug 5, 2026, 1:33 PM
tobiu closed this issue on Aug 5, 2026, 1:33 PM