Context
Successor to #16462, carrying the two residuals @neo-gpt-emmy recorded when she approved PR #16466 as Approve+Follow-Up (review).
That disposition was requested by @tobiu under an active firefighting threshold: the GPT family is near its rate limit, a client deployment is down, and the delivered core — unplanned restart churn becoming visible through the existing inspection path — is independently valuable now. Both residuals terminate at a non-authoritative, record-only fact whose actionClass is record, so neither can actuate recovery, and neither was a merge gate.
Recording them here so the approval's conditions survive the merge. Low priority by explicit operator direction — filed for durability, not for immediate pickup.
Live latest-open sweep: checked the latest 20 open issues at 2026-08-04T07:12:37Z; no equivalent found. A2A in-flight claim sweep over the 10 most recent messages (all read-states): no competing [lane-claim] on this scope.
The Problem
1. Planned-restart provenance reads a ledger the production restart path never writes
countPlannedRestarts sources planned restarts from the data-heal event ledger (readHealLedger, filtering type === 'restart' && status === 'attempt'). Production container-lifecycle restarts do not land there.
Emmy's delta challenge, quoted because it names the exact seam:
The test fixture appends type=restart/status=attempt through appendHealEvent, but the production container lifecycle path does not write that row. RecoveryActuatorService routes restart outcomes through finishAction → appendRecoveryRunState; Orchestrator's attempt/outcome heal-event writer serves the data-recovery actuator, whose HEAL_ACTIONS excludes restart.
So on a live plane the subtraction finds zero planned restarts and every planned restart is counted as unplanned churn. The unit test passes because the fixture writes the row the production path does not — the test and the production producer disagree about where the fact lives, and the fixture is the one that is wrong.
This is the same defect class the parent ticket was opened for: a test that cannot fail on the behaviour it claims to cover.
2. A damaged baseline suppresses the signal without marking the record degraded
readChurnBaseline distinguishes three outcomes — absent (null), unreadable/malformed ({unreadable: true}), and valid — and writeChurnBaseline logs at ERROR when the write fails. Neither reaches the published service record.
So a plane whose baseline file is corrupt or unwritable emits no churn diagnosis and publishes a record that looks indistinguishable from a healthy one. The suppression is correct (a damaged baseline must not re-anchor the counter and manufacture churn); the silence about it is not. An operator reading inspect_deployment cannot tell "no churn" from "churn detection is broken".
The Architectural Reality
ai/daemons/orchestrator/services/DeploymentStateBridgeService.mjs — countPlannedRestarts, readChurnBaseline, writeChurnBaseline, and the published service record they feed.
ai/daemons/orchestrator/services/RecoveryActuatorService.mjs — the canonical lifecycle restart producer; finishAction → appendRecoveryRunState.
recoveryRunStateStore — its durable outcome store, and the correct provenance for a planned restart.
healEventLedgerStore — serves the data-recovery actuator; its HEAL_ACTIONS excludes restart, which is why the current source is empty in production.
ai/daemons/orchestrator/services/ContainerHealthDiagnosisService.mjs — consumes the count; its restart-churn fact is authoritative: false with actionClass: record, and its classification branch is deliberately last. That is what bounds the blast radius of both residuals.
The Fix
- Source planned restarts from the canonical producer. Read
recoveryRunStateStore via readRecentRecoveryRunStates rather than the heal ledger, and key on the restart outcomes RecoveryActuatorService actually persists. Tests must drive the canonical producer/store instead of fabricating a row.
- Let a damaged baseline degrade the published record. Carry the
unreadable outcome and a write failure into the service record as an explicit degraded/unavailable marker on the churn section, so absence of a diagnosis is distinguishable from inability to produce one.
Contract Ledger Matrix
| Target Surface |
Source of Authority |
Proposed Behavior |
Fallback / Error Semantics |
Docs |
Evidence |
| planned-restart count |
recoveryRunStateStore via readRecentRecoveryRunStates |
Counts restarts the lifecycle actuator actually persisted |
Unreadable store ⇒ subtract nothing AND mark degraded; never silently zero |
ADR-0025 §2.4 |
a production restart is subtracted without a fabricated ledger row |
| churn record degradation |
readChurnBaseline / writeChurnBaseline outcomes |
Publishes a degraded marker when the baseline cannot be read or written |
Degraded is a positive statement, never an omission |
ADR-0025 §2.4 |
a corrupt baseline file yields a record an operator can distinguish from healthy |
Decision Record impact
aligned-with ADR 0025 — the restart-churn detect signal it already admits. No authority, actuator or action-class change: the fact stays authoritative: false with actionClass: record, per AC-7's recorded acceptance that the orchestrator is the actuator's only home.
Acceptance Criteria
Out of Scope
- Making the churn fact authoritative or letting it actuate recovery. ADR-0025 §2.1 separates detect-signal from actuator-authority and AC-7 records the single-home acceptance deliberately.
- Widening
HEAL_ACTIONS to include restart. The two ledgers serve different actuators; merging them to make one query convenient is the wrong direction.
- The L4 question of whether the churn thresholds are correct on a live plane — that belongs to #16463.
Avoided Traps
- Adding
restart to the heal ledger so the existing query works. That makes the data-recovery actuator's ledger carry lifecycle events it does not own, and the convenience would be paid for by every future reader that has to ask which actuator wrote a row.
- Treating a damaged baseline as zero churn. Re-anchoring on a damaged baseline is what #16462's
656bc225cc fixed; this ticket must not reintroduce it while making the failure visible.
Related
- Parent / delivered core: #16462 via PR #16466 (approved at
7ca8c370ea)
- Sibling L4 lane: #16463
- Governing decisions: ADR-0025 (detect), ADR-0026 (actuator)
Retrieval Hint: planned restart provenance recoveryRunStateStore vs heal ledger churn baseline degraded
Origin Session ID: 8132018f-e94a-4ef5-bce4-ec788ef56b7e
Context
Successor to #16462, carrying the two residuals @neo-gpt-emmy recorded when she approved PR #16466 as Approve+Follow-Up (review).
That disposition was requested by @tobiu under an active firefighting threshold: the GPT family is near its rate limit, a client deployment is down, and the delivered core — unplanned restart churn becoming visible through the existing inspection path — is independently valuable now. Both residuals terminate at a non-authoritative, record-only fact whose
actionClassisrecord, so neither can actuate recovery, and neither was a merge gate.Recording them here so the approval's conditions survive the merge. Low priority by explicit operator direction — filed for durability, not for immediate pickup.
Live latest-open sweep: checked the latest 20 open issues at 2026-08-04T07:12:37Z; no equivalent found. A2A in-flight claim sweep over the 10 most recent messages (all read-states): no competing
[lane-claim]on this scope.The Problem
1. Planned-restart provenance reads a ledger the production restart path never writes
countPlannedRestartssources planned restarts from the data-heal event ledger (readHealLedger, filteringtype === 'restart' && status === 'attempt'). Production container-lifecycle restarts do not land there.Emmy's delta challenge, quoted because it names the exact seam:
So on a live plane the subtraction finds zero planned restarts and every planned restart is counted as unplanned churn. The unit test passes because the fixture writes the row the production path does not — the test and the production producer disagree about where the fact lives, and the fixture is the one that is wrong.
This is the same defect class the parent ticket was opened for: a test that cannot fail on the behaviour it claims to cover.
2. A damaged baseline suppresses the signal without marking the record degraded
readChurnBaselinedistinguishes three outcomes — absent (null), unreadable/malformed ({unreadable: true}), and valid — andwriteChurnBaselinelogs at ERROR when the write fails. Neither reaches the published service record.So a plane whose baseline file is corrupt or unwritable emits no churn diagnosis and publishes a record that looks indistinguishable from a healthy one. The suppression is correct (a damaged baseline must not re-anchor the counter and manufacture churn); the silence about it is not. An operator reading
inspect_deploymentcannot tell "no churn" from "churn detection is broken".The Architectural Reality
ai/daemons/orchestrator/services/DeploymentStateBridgeService.mjs—countPlannedRestarts,readChurnBaseline,writeChurnBaseline, and the published service record they feed.ai/daemons/orchestrator/services/RecoveryActuatorService.mjs— the canonical lifecycle restart producer;finishAction → appendRecoveryRunState.recoveryRunStateStore— its durable outcome store, and the correct provenance for a planned restart.healEventLedgerStore— serves the data-recovery actuator; itsHEAL_ACTIONSexcludesrestart, which is why the current source is empty in production.ai/daemons/orchestrator/services/ContainerHealthDiagnosisService.mjs— consumes the count; itsrestart-churnfact isauthoritative: falsewithactionClass: record, and its classification branch is deliberately last. That is what bounds the blast radius of both residuals.The Fix
recoveryRunStateStoreviareadRecentRecoveryRunStatesrather than the heal ledger, and key on the restart outcomesRecoveryActuatorServiceactually persists. Tests must drive the canonical producer/store instead of fabricating a row.unreadableoutcome and a write failure into the service record as an explicit degraded/unavailable marker on the churn section, so absence of a diagnosis is distinguishable from inability to produce one.Contract Ledger Matrix
recoveryRunStateStoreviareadRecentRecoveryRunStatesreadChurnBaseline/writeChurnBaselineoutcomesDecision Record impact
aligned-with ADR 0025— the restart-churn detect signal it already admits. No authority, actuator or action-class change: the fact staysauthoritative: falsewithactionClass: record, per AC-7's recorded acceptance that the orchestrator is the actuator's only home.Acceptance Criteria
RecoveryActuatorServicewrites; a restart persisted throughfinishAction → appendRecoveryRunStateis subtracted.inspect_deployment.actionClass: record; no privilege or action boundary moves.Out of Scope
HEAL_ACTIONSto includerestart. The two ledgers serve different actuators; merging them to make one query convenient is the wrong direction.Avoided Traps
restartto the heal ledger so the existing query works. That makes the data-recovery actuator's ledger carry lifecycle events it does not own, and the convenience would be paid for by every future reader that has to ask which actuator wrote a row.656bc225ccfixed; this ticket must not reintroduce it while making the failure visible.Related
7ca8c370ea)Retrieval Hint:
planned restart provenance recoveryRunStateStore vs heal ledger churn baseline degradedOrigin Session ID: 8132018f-e94a-4ef5-bce4-ec788ef56b7e