Context
Measured on the canonical plane, 2026-08-09 ~15:26 CEST:
mc-server healthcheck interval=10s timeout=10s retries=12
State.Health.FailingStreak 4
State.Health.Status healthy ← what every surface reports
Health.Log tail 3× "Health check exceeded timeout (10s)"
1× "MCP healthcheck connect timed out [service-unresponsive]"Simultaneously, on two different maintainer seats, Memory Core tool calls were failing — timeouts on one, transport-level Streamable HTTP error on the other — including add_memory, which §critical_gates rule 5 makes the mandatory end-of-turn save. A peer failed it twice with a verified retry and burned 45 s working around it. That turn's memory is gone.
Every container reported healthy throughout.
The Problem
A container runtime flips to unhealthy only after retries CONSECUTIVE probe failures. At retries: 12 that is two unbroken minutes. An intermittent failure resets the streak on every success and therefore never accumulates twelve in a row — so a service failing a third of its probes, indefinitely, is structurally incapable of being marked unhealthy.
The layer can represent dead and healthy. It has no representation for degraded-but-serving, which is the state a saturated or contended plane actually occupies, and the state both affected seats worked in for hours.
A second asymmetry sits underneath it and is the subtler half: the probe is cheaper than the traffic. The healthcheck is a small healthcheck tool call; add_memory carries an entire turn. A probe lighter than the workload systematically under-detects the failure that matters, because it passes on exactly the calls cheap enough to succeed. This is #16691's family — "KB health reports operational without ever probing embedding" — one service over and one level less obvious: not an omitted subsystem, an unrepresentative sample.
Why this is not merely a tuning complaint. Lowering retries trades this for false positives during startup, which is what the high value was chosen to prevent, and that trade has already been made deliberately. The gap is not the threshold; it is that a binary derived from consecutiveness cannot express a rate, and degradation is a rate.
The Architectural Reality
- The evidence already exists and is already read.
DeploymentStateBridgeService performs a per-service inspect, and State.Health carries FailingStreak and a Health.Log ring of recent probe results with exit codes and timestamps. Neither reaches the snapshot; only the folded binary status does.
- So this is a projection gap rather than a collection gap — the same shape as
#16791 (the running revision exists in every container and is published nowhere).
- The recovery lane cannot cover this.
#16766's controller requires a container-unhealthy fact, which by construction is never created for an intermittently-failing service. Anything built on the binary inherits the blind spot; the fix has to be a signal that does not depend on consecutiveness.
- A rate is an observation, not a verdict. Publishing it does not imply acting on it, and this ticket deliberately stops at publishing.
Contract Ledger Matrix
| Target Surface |
Source of Authority |
Proposed Behavior |
Fallback / Edge Case |
Docs |
Evidence |
| deployment snapshot, per service |
existing inspect read |
Publish recent probe-failure rate and the current FailingStreak alongside the binary status |
No State.Health (no healthcheck declared) ⇒ explicit not-applicable, distinct from a zero rate |
Method JSDoc + snapshot contract |
Fixture: a log of mixed pass/fail yields a non-zero rate while status stays healthy |
existing status field |
container runtime |
unchanged |
— |
— |
Negative spec: the binary keeps its exact current values |
| snapshot size |
existing maxSnapshotBytes |
Two numbers per service |
bounded by construction |
— |
existing size guard |
Acceptance Criteria
Out of Scope
- Acting on the rate. No restart, no alarm, no recovery class.
#16766's controller consumes container-unhealthy and this ticket does not feed it — deliberately, because an action driven by a rate needs a threshold nobody has evidence for yet.
- Changing
retries or interval. The current values were chosen to avoid startup false-positives; this ticket does not relitigate that trade.
- Making the healthcheck exercise a heavier operation. That is a real and separate question — a probe representative of the workload — and it belongs with the service that owns the probe, not here.
- The Memory Core wedge itself (
#16677) and the mailbox repair that may already fix it.
Avoided Traps
- Reading a green binary as a healthy service. It was green through the entire observed incident, on a plane where the mandatory memory save was failing on two seats.
- Treating a zero rate and no-healthcheck as one state. A third-party image with no probe and a service passing every probe need opposite responses.
- Turning the observation into a verdict. The moment the plane says "degraded", it needs a threshold, and that threshold will be wrong for someone. Publish the rate; let a consumer judge.
- Assuming the recovery lane covers it.
#16766 cannot fire here — no container-unhealthy fact is ever produced. Anything derived from the binary inherits the same blindness.
Related
#16677 (the wedge whose detection this failed) · #16691 (the precedent: a status that means nothing is worse than no status) · #16766 (the controller that structurally cannot see this) · #16791 (the sibling projection gap — the datum exists and is published nowhere) · #16706 (the epic whose we saw it clause this sits under)
Origin Session ID: a641ddac-565a-4fc8-adc1-6c25629bddb7
Retrieval Hint: degraded but serving intermittent probe failure never reaches retries consecutive failing streak not published
Context
Measured on the canonical plane, 2026-08-09 ~15:26 CEST:
mc-server healthcheck interval=10s timeout=10s retries=12 State.Health.FailingStreak 4 State.Health.Status healthy ← what every surface reports Health.Log tail 3× "Health check exceeded timeout (10s)" 1× "MCP healthcheck connect timed out [service-unresponsive]"Simultaneously, on two different maintainer seats, Memory Core tool calls were failing — timeouts on one, transport-level
Streamable HTTP erroron the other — includingadd_memory, which§critical_gatesrule 5 makes the mandatory end-of-turn save. A peer failed it twice with a verified retry and burned 45 s working around it. That turn's memory is gone.Every container reported
healthythroughout.The Problem
A container runtime flips to
unhealthyonly afterretriesCONSECUTIVE probe failures. Atretries: 12that is two unbroken minutes. An intermittent failure resets the streak on every success and therefore never accumulates twelve in a row — so a service failing a third of its probes, indefinitely, is structurally incapable of being marked unhealthy.The layer can represent dead and healthy. It has no representation for degraded-but-serving, which is the state a saturated or contended plane actually occupies, and the state both affected seats worked in for hours.
A second asymmetry sits underneath it and is the subtler half: the probe is cheaper than the traffic. The healthcheck is a small
healthchecktool call;add_memorycarries an entire turn. A probe lighter than the workload systematically under-detects the failure that matters, because it passes on exactly the calls cheap enough to succeed. This is#16691's family — "KB health reports operational without ever probing embedding" — one service over and one level less obvious: not an omitted subsystem, an unrepresentative sample.Why this is not merely a tuning complaint. Lowering
retriestrades this for false positives during startup, which is what the high value was chosen to prevent, and that trade has already been made deliberately. The gap is not the threshold; it is that a binary derived from consecutiveness cannot express a rate, and degradation is a rate.The Architectural Reality
DeploymentStateBridgeServiceperforms a per-serviceinspect, andState.HealthcarriesFailingStreakand aHealth.Logring of recent probe results with exit codes and timestamps. Neither reaches the snapshot; only the folded binary status does.#16791(the running revision exists in every container and is published nowhere).#16766's controller requires acontainer-unhealthyfact, which by construction is never created for an intermittently-failing service. Anything built on the binary inherits the blind spot; the fix has to be a signal that does not depend on consecutiveness.Contract Ledger Matrix
inspectreadFailingStreakalongside the binary statusState.Health(no healthcheck declared) ⇒ explicitnot-applicable, distinct from a zero ratehealthystatusfieldmaxSnapshotBytesAcceptance Criteria
retriesconsecutively — reports a non-zero failure rate while its status is stillhealthy. This is the exact observed state and it must be representable.not-applicable, distinguishable from a healthy service at a zero rate. One value for both would recreate the class this ticket exists to close.statusfield's existing values are unchanged, asserted by spec.Out of Scope
#16766's controller consumescontainer-unhealthyand this ticket does not feed it — deliberately, because an action driven by a rate needs a threshold nobody has evidence for yet.retriesorinterval. The current values were chosen to avoid startup false-positives; this ticket does not relitigate that trade.#16677) and the mailbox repair that may already fix it.Avoided Traps
#16766cannot fire here — nocontainer-unhealthyfact is ever produced. Anything derived from the binary inherits the same blindness.Related
#16677(the wedge whose detection this failed) ·#16691(the precedent: a status that means nothing is worse than no status) ·#16766(the controller that structurally cannot see this) ·#16791(the sibling projection gap — the datum exists and is published nowhere) ·#16706(the epic whose we saw it clause this sits under)Origin Session ID: a641ddac-565a-4fc8-adc1-6c25629bddb7
Retrieval Hint:
degraded but serving intermittent probe failure never reaches retries consecutive failing streak not published