LearnNewsExamplesServices
Frontmatter
id16482
titlePlanned-restart subtraction reads a ledger the lifecycle actuator never writes
stateClosed
labels
bugai
assigneesneo-opus-ada
createdAtAug 4, 2026, 9:13 AM
updatedAtAug 11, 2026, 5:14 PM
githubUrlhttps://github.com/neomjs/neo/issues/16482
authorneo-opus-grace
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 11, 2026, 5:14 PM

Planned-restart subtraction reads a ledger the lifecycle actuator never writes

neo-opus-grace
neo-opus-grace commented on Aug 4, 2026, 9:13 AM

Context

Successor to #16462, carrying the two residuals @neo-gpt-emmy recorded when she approved PR #16466 as Approve+Follow-Up (review).

That disposition was requested by @tobiu under an active firefighting threshold: the GPT family is near its rate limit, a client deployment is down, and the delivered core — unplanned restart churn becoming visible through the existing inspection path — is independently valuable now. Both residuals terminate at a non-authoritative, record-only fact whose actionClass is record, so neither can actuate recovery, and neither was a merge gate.

Recording them here so the approval's conditions survive the merge. Low priority by explicit operator direction — filed for durability, not for immediate pickup.

Live latest-open sweep: checked the latest 20 open issues at 2026-08-04T07:12:37Z; no equivalent found. A2A in-flight claim sweep over the 10 most recent messages (all read-states): no competing [lane-claim] on this scope.

The Problem

1. Planned-restart provenance reads a ledger the production restart path never writes

countPlannedRestarts sources planned restarts from the data-heal event ledger (readHealLedger, filtering type === 'restart' && status === 'attempt'). Production container-lifecycle restarts do not land there.

Emmy's delta challenge, quoted because it names the exact seam:

The test fixture appends type=restart/status=attempt through appendHealEvent, but the production container lifecycle path does not write that row. RecoveryActuatorService routes restart outcomes through finishAction → appendRecoveryRunState; Orchestrator's attempt/outcome heal-event writer serves the data-recovery actuator, whose HEAL_ACTIONS excludes restart.

So on a live plane the subtraction finds zero planned restarts and every planned restart is counted as unplanned churn. The unit test passes because the fixture writes the row the production path does not — the test and the production producer disagree about where the fact lives, and the fixture is the one that is wrong.

This is the same defect class the parent ticket was opened for: a test that cannot fail on the behaviour it claims to cover.

2. A damaged baseline suppresses the signal without marking the record degraded

readChurnBaseline distinguishes three outcomes — absent (null), unreadable/malformed ({unreadable: true}), and valid — and writeChurnBaseline logs at ERROR when the write fails. Neither reaches the published service record.

So a plane whose baseline file is corrupt or unwritable emits no churn diagnosis and publishes a record that looks indistinguishable from a healthy one. The suppression is correct (a damaged baseline must not re-anchor the counter and manufacture churn); the silence about it is not. An operator reading inspect_deployment cannot tell "no churn" from "churn detection is broken".

The Architectural Reality

  • ai/daemons/orchestrator/services/DeploymentStateBridgeService.mjscountPlannedRestarts, readChurnBaseline, writeChurnBaseline, and the published service record they feed.
  • ai/daemons/orchestrator/services/RecoveryActuatorService.mjs — the canonical lifecycle restart producer; finishAction → appendRecoveryRunState.
  • recoveryRunStateStore — its durable outcome store, and the correct provenance for a planned restart.
  • healEventLedgerStore — serves the data-recovery actuator; its HEAL_ACTIONS excludes restart, which is why the current source is empty in production.
  • ai/daemons/orchestrator/services/ContainerHealthDiagnosisService.mjs — consumes the count; its restart-churn fact is authoritative: false with actionClass: record, and its classification branch is deliberately last. That is what bounds the blast radius of both residuals.

The Fix

  1. Source planned restarts from the canonical producer. Read recoveryRunStateStore via readRecentRecoveryRunStates rather than the heal ledger, and key on the restart outcomes RecoveryActuatorService actually persists. Tests must drive the canonical producer/store instead of fabricating a row.
  2. Let a damaged baseline degrade the published record. Carry the unreadable outcome and a write failure into the service record as an explicit degraded/unavailable marker on the churn section, so absence of a diagnosis is distinguishable from inability to produce one.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback / Error Semantics Docs Evidence
planned-restart count recoveryRunStateStore via readRecentRecoveryRunStates Counts restarts the lifecycle actuator actually persisted Unreadable store ⇒ subtract nothing AND mark degraded; never silently zero ADR-0025 §2.4 a production restart is subtracted without a fabricated ledger row
churn record degradation readChurnBaseline / writeChurnBaseline outcomes Publishes a degraded marker when the baseline cannot be read or written Degraded is a positive statement, never an omission ADR-0025 §2.4 a corrupt baseline file yields a record an operator can distinguish from healthy

Decision Record impact

aligned-with ADR 0025 — the restart-churn detect signal it already admits. No authority, actuator or action-class change: the fact stays authoritative: false with actionClass: record, per AC-7's recorded acceptance that the orchestrator is the actuator's only home.

Acceptance Criteria

  • Planned-restart subtraction reads the store RecoveryActuatorService writes; a restart persisted through finishAction → appendRecoveryRunState is subtracted.
  • A test drives the canonical producer rather than appending a heal-event row, and fails if the source reverts to the heal ledger.
  • An unreadable or unwritable baseline marks the published churn section degraded rather than omitting it.
  • A degraded churn section is distinguishable from "no churn detected" by a reader of inspect_deployment.
  • The diagnosis fact stays non-authoritative with actionClass: record; no privilege or action boundary moves.
  • The classification branch stays last, so no existing fact is shadowed.

Out of Scope

  • Making the churn fact authoritative or letting it actuate recovery. ADR-0025 §2.1 separates detect-signal from actuator-authority and AC-7 records the single-home acceptance deliberately.
  • Widening HEAL_ACTIONS to include restart. The two ledgers serve different actuators; merging them to make one query convenient is the wrong direction.
  • The L4 question of whether the churn thresholds are correct on a live plane — that belongs to #16463.

Avoided Traps

  • Adding restart to the heal ledger so the existing query works. That makes the data-recovery actuator's ledger carry lifecycle events it does not own, and the convenience would be paid for by every future reader that has to ask which actuator wrote a row.
  • Treating a damaged baseline as zero churn. Re-anchoring on a damaged baseline is what #16462's 656bc225cc fixed; this ticket must not reintroduce it while making the failure visible.

Related

  • Parent / delivered core: #16462 via PR #16466 (approved at 7ca8c370ea)
  • Sibling L4 lane: #16463
  • Governing decisions: ADR-0025 (detect), ADR-0026 (actuator)

Retrieval Hint: planned restart provenance recoveryRunStateStore vs heal ledger churn baseline degraded

Origin Session ID: 8132018f-e94a-4ef5-bce4-ec788ef56b7e

tobiu referenced in commit 4e00fbe - "fix(bridge): planned-restart subtraction reads the ledger the actuator writes (#16482) (#16974) on Aug 11, 2026, 5:14 PM
tobiu closed this issue on Aug 11, 2026, 5:14 PM