LearnNewsExamplesServices
Frontmatter
id16695
titleA heap ceiling cannot be applied without recreating the container
stateOpen
labels
enhancementaiarchitectureagent-os
assigneesneo-opus-vega
createdAtAug 8, 2026, 3:35 PM
updatedAtAug 10, 2026, 10:40 AM
githubUrlhttps://github.com/neomjs/neo/issues/16695
authorneo-opus-vega
commentsCount18
parentIssue16706
subIssues
16857 The prescription renderer has no caller and no admission boundary
16868 Deliver registry-valid prescriptions through the host pipeline
subIssuesCompleted2
subIssuesTotal2
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]

A heap ceiling cannot be applied without recreating the container

Open Backlog/active-chunk-14 enhancementaiarchitectureagent-os
neo-opus-vega
neo-opus-vega commented on Aug 8, 2026, 3:35 PM

Successor holding the two service-heap ceiling descriptors withdrawn from PR #16663 under @neo-gpt-emmy's Drop+Supersede, plus the delivery question that made them undeliverable.

Context

#16636 wrote kb-server-heap-ceiling and mc-server-heap-ceiling into RECOVERY_KNOBS and routed them at reconfigure. Review established that the channel cannot carry the payload, so the descriptors were withheld rather than merged. They are good descriptors with no way to take effect.

The Problem

--max-old-space-size lives in the container's command, and compose interpolates it at create time:

<h1 class="neo-h1" data-record-id="4">ai/deploy/docker-compose.yml:117</h1>

command: ["sh", "-c", "node --max-old-space-size=${NEO_KB_SERVER_HEAP_MB:-768} \"$SERVER_ENTRYPOINT\""]

reconfigure is overlay-plus-restart: RecoveryActuatorService.mjs:586restartComposeServiceapplyLifecycle({operation: 'restart'}). A restart re-runs the baked Config.Cmd, so the new value never reaches the process.

The failure shape is what makes this urgent rather than merely wrong: prescribing reconfigure would have produced a no-op that reports success. The diagnosis reads correct, the recorded action reads correct, and the ceiling never moves — which is indistinguishable from a working heal until the service dies at the old ceiling.

There is also no writer at all today — and the original statement of that fact here was wrong on both of its file citations, corrected 2026-08-09 rather than left for an implementer to trip over. I wrote that heapCeilingMb "occurs only in recoveryKnobRegistry.mjs and DeploymentStateBridgeService.mjs". Re-measured at dev: the exact-case identifier heapCeilingMb appears in neither — only declaredHeapCeilingMb exists, in the bridge and the diagnosis service, and it is a parsed observation of the container command, not a knob. And recoveryKnobRegistry.mjs is not at the path I gave; it lives at ai/services/memory-core/helpers/.

The conclusion survives and is in fact stronger than I stated: the descriptors were withdrawn from PR #16663 and never landed anywhere, so there is no knob field to read or write — not a field nobody consumes. Recorded because I nearly read my own empty grep as confirmation; the positive control (does the cited file exist at the cited path?) is what caught it.

The Architectural Reality

  • Applying the value requires container recreation (compose up -d with the new environment), not restart. That is a materially larger privilege than reconfigure and closer to redeploy.
  • Recreation carries the harm raise-ceiling exists to avoid: a service recreated mid-work loses in-flight state. Whatever this becomes must say what it refuses to disturb.
  • The descriptors themselves are ready and were reviewed: MB units, relational bounds, no constants, resource: 'v8-heap', a non-heap-RSS headroom bound, and a raise-not-lower predicate that compares against the current declaration and fails closed on unresolved or diverging ('unknown') observations.

The Fix

Answer the delivery question first; the descriptors land only if it has an acceptable answer.

  1. Determine whether a recreation-class action is acceptable in the actuator's authority envelope, with §2.4 accounting. Candidate shapes: an amendment to redeploy, a new bounded class, or an operator-gated path that is prescribed but never autonomously actuated.
  2. If yes, give it a contract: what it recreates, what it must not disturb, what it reports when recreation succeeds but the process does not return.
  3. Then land the descriptors and their tests against that class.

A negative answer closes this ticket legitimately. The observability half already shipped without them, and a knob nothing can turn is the defect #16636's own finding 1 names in throttle-shed.

Acceptance Criteria

  • The delivering action class is decided, with its ADR-0026 §2.4 accounting stated — including amends if the class is new or widened.
  • If accepted: the class has an explicit contract covering recreation scope, the state it must not disturb, and its failure reporting.
  • If accepted: the two descriptors and serviceHeapCeilingKnob.spec.mjs land against that class, carrying the reviewed predicates unchanged.
  • The prescription is proven to reach an effect, not merely to be recorded — a fixture that asserts only "a diagnosis named this action" would have passed against reconfigure throughout, which is exactly how the original defect survived to review.
  • Negative control: a service already at or above the requested ceiling is not prescribed, and an unresolved or diverging declaration refuses rather than raising against a guess.
  • If rejected: the rejection is recorded with the evidence, and #16636's re-opened knob AC is closed as withdrawn rather than left dangling.

Out of Scope

  • The prescription placement question — #16676 owns that, and it now depends on this ticket's answer.
  • The reactive controller for any action class.
  • Chroma's store-side ceiling — #16595 / #16596 / #16637 — which uses a live cgroup update precisely because it must not restart.

Avoided Traps

Assuming a restart re-reads the composition. This is the error that produced the original defect, and it is easy to repeat: reconfigure genuinely does write a durable overlay, so the mutation is real — only its arrival at the process is not. Any future channel claim must name the mechanism by which the value reaches Config.Cmd.

Treating "recorded a prescription" as delivery. See AC-4.

Related

#16636 (origin — its knob AC is re-opened and moves here) · #16663 (closed unmerged; the Drop+Supersede carries the salvage map) · #16676 (placement, blocked on this) · #16630 · ADR-0026 §2.4

Live latest-open sweep: checked latest 12 open issues at 2026-08-08T13:33:56Z; no equivalent found. A2A sweep: 12 most recent messages, no overlapping claim.

Origin Session ID: 4141258c-36d3-4788-b0c2-ab3ebe0867be

Retrieval Hint: query_raw_memories("heap ceiling needs container recreation, restart reuses Config.Cmd, no-op reporting success")

tobiu referenced in commit d589d8c - "feat(orchestrator): the crash diagnosis names a heap abort instead of a generic crash (#16750) (#16751) on Aug 9, 2026, 2:06 AM