Split from #16630 Slice B as the honest one-PR leaf. #16630 remains open for the predictive half (a V8-scoped numerator via in-process perf_hooks GC observation) — this ticket delivers only the reactive attribution, which is separable, already implementable, and needs no new channel.
Design work on #16630 this evening produced three measured findings that make this leaf worth filing on its own:
The container-scoped detector cannot fire ahead of a heap death. Both MCP servers run --max-old-space-size=768 under a 1 GiB limit at memorySaturationPercent: 90, so memory-saturation needs 921.6 MiB while old space dies at 768 MiB — it requires ≥153.6 MiB of non-old-space memory to warn at all. Its sensitivity to a heap death is a function of non-heap memory.
There is no IN-PROCESS actuator — v8.setFlagsFromString('--max-old-space-size=N') is accepted and inert (three-arm receipt on #16695, by @neo-opus-grace: reported limit 272 → 272, process aborts identically to the no-flag control). An earlier revision of this ticket generalized that to "no live actuator", which is false and was retracted by its own author: ai/examples/cloud-deployment/deploy-pipeline.sh is a governed recreation channel and has already delivered heap raises in production (#16463 at 2048, PR #16558 persisting 6144). The inert result bounds the in-process layer only.
But the death IS observable after the fact — from the log line, not the exit code. On the canonical neo-local-agent-os-mc-server image (Node v24.16.0, bounded cgroup) a heap death emits FATAL ERROR: … JavaScript heap out of memory and exits 139 with OOMKilled false — in both the declared and undeclared arms (@neo-gpt-emmy). An earlier revision of this ticket recorded 134/0 from a host Node and built a sequencing story on the difference; that was the wrong subject and both claims are retired. The exit code is not even stable across base images, which is the argument against keying attribution to it at all.
Live latest-open sweep: checked the latest 20 open issues at 2026-08-08T20:5xZ; no equivalent found. A2A in-flight claim sweep over the last 30 messages (all read-states): no competing claim on this surface.
The Problem
A service that exhausts its declared V8 ceiling already produces an authoritative container-down fact, already classifies as recoveryClass: 'crash', and is already restarted. The action was never wrong. What is absent is the cause: nothing reads the exit code, so the death records as a generic lifecycle-crash, the ceiling is never implicated, and the identical abort recurs indefinitely.
That is precisely why nothing looks broken from the outside — the heal succeeds every time, and the thing that would tell an operator to raise the ceiling is never emitted.
The Architectural Reality
ContainerHealthDiagnosisService.collectLifecycleFacts emits the container-down fact and its detailsalready carry exitCode — the raw signal is present and unread.
The crash branch returns reason: 'lifecycle-crash' for every non-running status, regardless of how the process died.
DeploymentStateBridgeService independently reads state.OOMKilled, so the field is available at the inspect layer; the diagnosis fact simply does not carry it.
#16630's Out of scope forbids "a second heap diagnosis kind beside memory-saturation — one signal needs the right numerator, not a sibling free to disagree." This ticket is an enrichment of an existing authoritative fact, so that line stands unamended.
The Fix
Carry the discriminating evidence on the fact, and narrow the crash reason only when the evidence supports it:
classifyHeapExhaustion({logs, nodeCommand, declaredHeapCeilingMb}) — tri-state, returning the attribution plus an unavailableReason.
container-down details gain oomKilled (raw evidence) and the attribution fields.
The crash branch reports lifecycle-crash-heap-exhaustion[-declared-ceiling] when the evidence names a heap, and lifecycle-crash otherwise. The action class does not change.
The bridge summarizes inspect and logs once and passes {logs, nodeCommand, declaredHeapCeilingMb} to diagnose() — no parsing is duplicated in diagnosis.
It is a VERDICT, because the evidence names a heap. An earlier revision emitted a candidate from the exit code — a compromise forced by a capability gap that turned out not to exist. The strict fatal line says heap; nodeCommand says the process had one to exhaust. No abort code carries heap semantics: it names the manner of death only, and every V8 FATAL ERROR, assertion failure and explicit abort() shares it.
Tri-state is load-bearing: null means the deciding fields were not observed and must read as absence of signal, never as a negative. A missing ExitCode is not evidence of a healthy exit.
true only from the producer receipt and a logs/inspect target match
false — never inferred, never caller-supplied
inline
spec: a different-container read never publishes bounded
Decision Record impact
aligned-with ADR 0026 — the §2.4 apply(serviceKey, action) interface and the detect ÷ actuator separation are untouched; no action class is added, removed, or re-routed. aligned-with ADR 0025 — this enriches a diagnosis fact rather than changing what the diagnosis decides.
Acceptance Criteria
Rewritten 2026-08-08 ~21:5xZ. The original criteria specified an exit-code discriminator (ExitCode === 134). That was falsified during review: the canonical neo-local-agent-os-mc-server image exits 139 where a host Node exits 134, and it exits 139 whether or not a ceiling was declared — so the code neither ports nor discriminates. The criteria below specify the evidence that actually names a heap. Falsification and the B-prime scoping correction: @neo-gpt-emmy on PR #16751.
Attribution requires both the strict fatal line (FATAL ERROR: … JavaScript heap out of memory) andnodeCommand === true. Either alone must not attribute.
The declared ceiling is not part of the discriminator. An undeclared Node service that dies of a heap still attributes — that population is where this class was first observed, and scoping to the ceiling would blind the detector to it. A numeric declaredHeapCeilingMb licenses only the stronger wording.
A non-Node service never attributes, whatever its log tail contains — proven by a red control, since a tail can carry another container's output or a service can simply log the phrase.
Unavailable is null with a stated reason, never false: logs disabled, an unreadable command, and a truncated non-matching tail each yield null plus an unavailableReason. A disabled channel is not evidence of health.
A truncated tail that DOES match stays conclusive — truncation cannot manufacture the line.
The container-down fact carries exitCode and oomKilled as raw evidence alongside the attribution — recorded, but not attribution inputs.
The crash diagnosis reports lifecycle-crash-heap-exhaustion-declared-ceiling, lifecycle-crash-heap-exhaustion, or lifecycle-crash, matching the evidence available.
The action class is unchanged in every branch — attribution must not alter the heal.
The scoping guard is shown capable of failing: removing the nodeCommand requirement reds the non-Node red control.
No parsing is duplicated in diagnosis.nodeCommand and declaredHeapCeilingMb are consumed from the bridge's existing Config.Cmd observations, and the log tail from its existing allowlisted read.
The log slice is bound to the stopped incarnation by an EXACT interval.since = State.StartedAt, until = State.FinishedAt, both transmitted unrounded — flooring the upper edge discards the final sub-second in which V8 writes its fatal line, and flooring the lower edge reaches into the previous run.
The bound proves its TARGET, not just a time range.readObserve resolves a target per call, so incarnationBounded requires the logs read and the inspect to have resolved the same container. A matching range on the wrong container is not this incarnation.
Either bound missing, invalid, or half-open ⇒ unbounded, and the classifier keeps refusing with log-incarnation-unbounded. Docker's unset time is the zero instant, which parses valid — it must be refused rather than passed through.
Out of Scope
The V8-scoped numerator (#16630 Slice B AC-1) — the predictive perf_hooks GC-observer channel. This ticket delivers attribution, not saturation.
Any change to memory-saturation or its threshold — the numerator moves scope there, not here.
A new log-read capability — retired as false.DeploymentStateBridgeService:392 already performs the allowlisted read and :449 publishes a bounded tail, so no new capability or privilege is involved. Asserting this boundary without checking it is what justified the weaker exit-code design.
Raising a ceiling in response — #16695 establishes only that no in-process channel exists; a governed recreation channel does, and the actuator question is @neo-opus-grace's lane.
Avoided Traps
Attributing on the exit code at all. An earlier revision of this ticket said 134 was "necessary and not sufficient" — the canonical image falsified even the necessary half (it exits 139, in both the declared and undeclared arms). No abort code carries heap semantics: it names the manner of death, and its value drifts with the base image. A detector keyed to one would both misattribute other abort classes and miss the real thing.
Matching the fatal line anywhere in the tail. The bounded log read spans container restarts, so an old heap-fatal line above a healthy boot above an unrelated crash attributes the current death as heap. The evidence window must be bound to the stopped incarnation — the same wider-than-the-thing-it-names defect #16500 was opened for.
A boolean instead of a tri-state.false for an unobserved field would assert "not a heap abort" from missing data, which is the fabricated-verdict shape the presence-contract work on D#16720 rejects on a different substrate.
A new fact type. A sibling diagnosis free to disagree with memory-saturation is exactly what #16630 excludes; enrichment keeps one signal.
Changing the action. Restart is already correct for a stopped container; a "fix" that alters the heal would be trading a real repair for a label.
Related
#16630 (parent scope — Slice B's predictive half stays open there) · #16695 (no in-process V8-ceiling actuator; a governed recreation channel exists — @neo-opus-grace) · #16636 (service-side ceiling routing) · #16642 / PR #16640 (Slice A, which produced the signature this reads) · ADR-0025 / ADR-0026
tobiu referenced in commit d589d8c - "feat(orchestrator): the crash diagnosis names a heap abort instead of a generic crash (#16750) (#16751) on Aug 9, 2026, 2:06 AM
Context
Split from
#16630Slice B as the honest one-PR leaf.#16630remains open for the predictive half (a V8-scoped numerator via in-processperf_hooksGC observation) — this ticket delivers only the reactive attribution, which is separable, already implementable, and needs no new channel.Design work on
#16630this evening produced three measured findings that make this leaf worth filing on its own:--max-old-space-size=768under a 1 GiB limit atmemorySaturationPercent: 90, somemory-saturationneeds 921.6 MiB while old space dies at 768 MiB — it requires ≥153.6 MiB of non-old-space memory to warn at all. Its sensitivity to a heap death is a function of non-heap memory.v8.setFlagsFromString('--max-old-space-size=N')is accepted and inert (three-arm receipt on#16695, by @neo-opus-grace: reported limit 272 → 272, process aborts identically to the no-flag control). An earlier revision of this ticket generalized that to "no live actuator", which is false and was retracted by its own author:ai/examples/cloud-deployment/deploy-pipeline.shis a governed recreation channel and has already delivered heap raises in production (#16463at 2048, PR#16558persisting 6144). The inert result bounds the in-process layer only.neo-local-agent-os-mc-serverimage (Nodev24.16.0, bounded cgroup) a heap death emitsFATAL ERROR: … JavaScript heap out of memoryand exits139withOOMKilledfalse — in both the declared and undeclared arms (@neo-gpt-emmy). An earlier revision of this ticket recorded134/0from a host Node and built a sequencing story on the difference; that was the wrong subject and both claims are retired. The exit code is not even stable across base images, which is the argument against keying attribution to it at all.Live latest-open sweep: checked the latest 20 open issues at 2026-08-08T20:5xZ; no equivalent found. A2A in-flight claim sweep over the last 30 messages (all read-states): no competing claim on this surface.
The Problem
A service that exhausts its declared V8 ceiling already produces an authoritative
container-downfact, already classifies asrecoveryClass: 'crash', and is already restarted. The action was never wrong. What is absent is the cause: nothing reads the exit code, so the death records as a genericlifecycle-crash, the ceiling is never implicated, and the identical abort recurs indefinitely.That is precisely why nothing looks broken from the outside — the heal succeeds every time, and the thing that would tell an operator to raise the ceiling is never emitted.
The Architectural Reality
ContainerHealthDiagnosisService.collectLifecycleFactsemits thecontainer-downfact and itsdetailsalready carryexitCode— the raw signal is present and unread.reason: 'lifecycle-crash'for every non-running status, regardless of how the process died.DeploymentStateBridgeServiceindependently readsstate.OOMKilled, so the field is available at the inspect layer; the diagnosis fact simply does not carry it.#16630's Out of scope forbids "a second heap diagnosis kind besidememory-saturation— one signal needs the right numerator, not a sibling free to disagree." This ticket is an enrichment of an existing authoritative fact, so that line stands unamended.The Fix
Carry the discriminating evidence on the fact, and narrow the crash reason only when the evidence supports it:
classifyHeapExhaustion({logs, nodeCommand, declaredHeapCeilingMb})— tri-state, returning the attribution plus anunavailableReason.container-downdetails gainoomKilled(raw evidence) and the attribution fields.lifecycle-crash-heap-exhaustion[-declared-ceiling]when the evidence names a heap, andlifecycle-crashotherwise. The action class does not change.{logs, nodeCommand, declaredHeapCeilingMb}todiagnose()— no parsing is duplicated in diagnosis.It is a VERDICT, because the evidence names a heap. An earlier revision emitted a candidate from the exit code — a compromise forced by a capability gap that turned out not to exist. The strict fatal line says heap;
nodeCommandsays the process had one to exhaust. No abort code carries heap semantics: it names the manner of death only, and every V8FATAL ERROR, assertion failure and explicitabort()shares it.Tri-state is load-bearing:
nullmeans the deciding fields were not observed and must read as absence of signal, never as a negative. A missingExitCodeis not evidence of a healthy exit.Contract Ledger Matrix
ContainerHealthDiagnosisServicecontainer-downfactdetailsexitCodeoomKilledas raw evidence +{heapExhaustion, unavailableReason, declaredHeapCeilingMb}nullwith a reason, never a negativeclassifyHeapExhaustion({logs, nodeCommand, declaredHeapCeilingMb})(new export)nodeCommandnull+unavailableReasonwhen the question cannot be askedHEAP_FATAL_LINE(new export)diagnose()inputsDeploymentStateBridgeService:392/:1122-1123{logs, nodeCommand, declaredHeapCeilingMb}from the bridge's existing observationsnulleach, degrading to unavailableclassificationReasonADR-0026§2.4 actuator matrixlifecycle-crash-heap-exhaustion[-declared-ceiling]; action class unchangedlifecycle-crashDeploymentRuntimeAccessService.readObserve({operation:'logs'})since/until; narrows an existing read, adds no capability, operation, or privilegebounded: falsedata)appliedSince/appliedUntilat full RFC3339 precision +containerIdnulleach;bounded:falsesummarizeLogs()→logs.incarnationBoundedDeploymentStateBridgeServicetrueonly from the producer receipt and a logs/inspect target matchfalse— never inferred, never caller-suppliedDecision Record impact
aligned-with ADR 0026— the §2.4apply(serviceKey, action)interface and the detect ÷ actuator separation are untouched; no action class is added, removed, or re-routed.aligned-with ADR 0025— this enriches a diagnosis fact rather than changing what the diagnosis decides.Acceptance Criteria
FATAL ERROR: … JavaScript heap out of memory) andnodeCommand === true. Either alone must not attribute.declaredHeapCeilingMblicenses only the stronger wording.nullwith a stated reason, neverfalse: logs disabled, an unreadable command, and a truncated non-matching tail each yieldnullplus anunavailableReason. A disabled channel is not evidence of health.container-downfact carriesexitCodeandoomKilledas raw evidence alongside the attribution — recorded, but not attribution inputs.lifecycle-crash-heap-exhaustion-declared-ceiling,lifecycle-crash-heap-exhaustion, orlifecycle-crash, matching the evidence available.nodeCommandrequirement reds the non-Node red control.nodeCommandanddeclaredHeapCeilingMbare consumed from the bridge's existingConfig.Cmdobservations, and the log tail from its existing allowlisted read.since=State.StartedAt,until=State.FinishedAt, both transmitted unrounded — flooring the upper edge discards the final sub-second in which V8 writes its fatal line, and flooring the lower edge reaches into the previous run.readObserveresolves a target per call, soincarnationBoundedrequires the logs read and the inspect to have resolved the same container. A matching range on the wrong container is not this incarnation.log-incarnation-unbounded. Docker's unset time is the zero instant, which parses valid — it must be refused rather than passed through.Out of Scope
#16630Slice B AC-1) — the predictiveperf_hooksGC-observer channel. This ticket delivers attribution, not saturation.memory-saturationor its threshold — the numerator moves scope there, not here.A new log-read capability— retired as false.DeploymentStateBridgeService:392already performs the allowlisted read and:449publishes a bounded tail, so no new capability or privilege is involved. Asserting this boundary without checking it is what justified the weaker exit-code design.#16695establishes only that no in-process channel exists; a governed recreation channel does, and the actuator question is @neo-opus-grace's lane.Avoided Traps
134was "necessary and not sufficient" — the canonical image falsified even the necessary half (it exits139, in both the declared and undeclared arms). No abort code carries heap semantics: it names the manner of death, and its value drifts with the base image. A detector keyed to one would both misattribute other abort classes and miss the real thing.#16500was opened for.falsefor an unobserved field would assert "not a heap abort" from missing data, which is the fabricated-verdict shape the presence-contract work onD#16720rejects on a different substrate.memory-saturationis exactly what#16630excludes; enrichment keeps one signal.Related
#16630(parent scope — Slice B's predictive half stays open there) ·#16695(no in-process V8-ceiling actuator; a governed recreation channel exists — @neo-opus-grace) ·#16636(service-side ceiling routing) ·#16642/ PR#16640(Slice A, which produced the signature this reads) ·ADR-0025/ADR-0026Origin Session ID: 46485b3a-9468-4df8-9390-3c961bd0e72f
Retrieval Hint:
query_raw_memories("heap abort exit 134 SIGABRT candidate not verdict container-down attribution undeclared ceiling exits 0 no signature")Authored by Vega (@neo-opus-vega, Claude Opus 5, Claude Code).