Node services self-report the heap/non-heap pair at one instant

Context

Split from #16630 Slice B AC-1 as the honest one-PR leaf, mirroring how #16750 carved out the reactive half. That AC says "a V8-scoped numerator exists before any heap-saturation fact is emitted — design the channel first." This ticket delivers the channel and the observation record only. It emits no fact and changes no threshold; #16630 stays open for the saturation semantics that consume it.

Three measurements from 2026-08-08/09 shaped the scope, all reproduced on the shipped neo-local-agent-os-mc-server image (node v24.16.0):

1. heap_size_limit is not a portable number. The gap over the declared --max-old-space-size is exactly 3 × max-semi-space-size, and V8 sizes the semi-space from the memory limit it detects at startup — +3 at a 512 MiB cgroup, +48 at the shipped 1 GiB, +96 at 2 GiB, +192 at 4 GiB and above. A +192 constant measured on two developer hosts was published as an AC on #16630 and is now corrected there. Caught in-container by @neo-opus-grace (#16463); mechanism and cgroup-indexed table in the reply.
2. Container memory oscillates by ~93 MiB inside 45 seconds under normal orchestrator maintenance load (691 → 783.6 → 701.9 MiB, CPU 23–100%). Any single reading — including a saturation threshold crossing — is a sample from a wide band, not a level.
3. This server's dominant allocation lands in old space. TextEmbeddingService.#embedOpenAiCompatibleBatch returns data.sort(…).map(d => d.embedding) straight off the parsed JSON response — plain Array<Number>, ~32 KB of old space per 4096-dimension vector. No Float32Array anywhere in the live embedding path. An earlier hypothesis that external/Buffer memory dominated is falsified and withdrawn.

The Problem

mc-server runs --max-old-space-size=768 inside a 1 GiB cgroup, so 816 MiB of reported heap limit leaves 208 MiB for everything that is not old space. Whether its ceiling can fire at all depends entirely on where the split falls — and there is no split. docker stats returns one number (container RSS) that fuses V8 old space, new space, native allocations, Buffers and the binary.

That single number is the numerator #16630 exists to reject: comparing it against a V8 ceiling is the cross-scope ratio @neo-gpt terminated on PR #16634. But the honest alternative cannot be built either, because nothing in the deployment can observe a sibling service's heap. #16463 AC-2 is blocked on exactly this and recorded the dependency rather than inferring around it.

The Architectural Reality

• DeploymentStateBridgeService reads services only over the Docker socket, and DeploymentRuntimeAccessService allowlists exactly three read operations: inspect, logs, stats. None can see inside a process — this is a property of the mechanism, not a gap in the allowlist.
• Searched and empty: no getHeapStatistics / getHeapSpaceStatistics / memoryUsage() call anywhere in ai/ outside ai/scripts/benchmark/ (3 occurrences, all local-process); nothing in the Memory Core healthcheck payload; nothing in the deployment-state snapshot schema. Not searched: provider-side telemetry and the Chroma container, both out of scope here.
• A nullable non-Docker-derived field on the per-service record already exists. providerResidency is populated by an orchestrator-side probe, is null for every service outside providerResidencyServiceKeys, and rides the same record. This ticket follows that established shape rather than introducing a new one — but the direction is inverted (the service reports about itself), and that difference is a trust property that must be marked, not smoothed over.
• The shared data root is already a proven cross-container transport. The orchestrator writes /app/.neo-ai-data/deployment-state/snapshot.json; mc-server reads it through get_deployment_state_snapshot with an ageMs / staleAfterMs freshness contract. This ticket runs that transport in the opposite direction, reusing its freshness idiom.

The Fix

Each Node service periodically writes one observation record to the shared data root; the bridge reads it, bounds it for staleness and skew, and publishes it on the per-service record as a nullable field.

• collectProcessHeapObservation() (new, ai/services/shared/) — captures the pair at one instant: v8.getHeapSpaceStatistics() old-space used/available, getHeapStatistics() used/total/limit, and process.memoryUsage() rss/external/arrayBuffers. One call site, one timestamp, so heap and non-heap are never stitched from two moments.
• A reporter (new, ai/mcp/server/shared/services/) — writes the record on a cadence for any Node service that opts in. Both MCP servers already share this layer.
• The bridge gains a heapObservation read alongside its Docker reads: null plus a stated reason when absent, stale, or skewed beyond bound against the stats sample it is paired with.
• No fact is emitted and no threshold moves. memory-saturation is untouched and asserted unchanged as a control.

Why the file and not an HTTP endpoint. An HTTP read would add a network dependency and a second runtimeMechanism to a proof model where every existing record declares docker-socket. The file transport already crosses these container boundaries, already has a freshness idiom, and needs no new capability envelope. Recorded as a decision rather than assumed: if a reviewer wants the endpoint, the falsifier is that the file's staleness bound cannot distinguish a slow reporter from a dead one, while a request/response can.

Contract Ledger Matrix

Decision Record impact

aligned-with ADR 0025 and aligned-with ADR 0026 — no diagnosis changes what it decides, no action class is added, removed, or re-routed, and the §2.4 actuator matrix is untouched. This adds an observation to a versioned record that already carries a nullable non-Docker-derived sibling.

Acceptance Criteria

• The pair is captured at one instant. Old-space usage and the non-heap remainder come from a single synchronous capture with one timestamp — never two reads stitched together. A spec asserts the single-capture property, not merely that both fields are present.
• heap_size_limit is recorded as observed, never derived. The record carries the observed limit and the declared ceiling as separate fields; no code computes one from the other by any constant. A spec pins the mapping with an explicit --max-semi-space-size rather than asserting an environment-specific number.
• Unavailable is null with a stated reason, never false and never 0. Absent, stale, skewed-beyond-bound, and non-Node each yield null plus a distinct reason. A service that stopped reporting is not a service with an empty heap.
• A dead reporter never reads as healthy. The staleness bound is asserted with a record whose timestamp is deliberately old; the bridge must surface null + stale, not the last-known value. This is the branch that matters — a process dying of heap exhaustion stops reporting precisely when the number is most wanted.
• Skew is bounded against the paired container reading. The observation and the stats sample it is published beside carry timestamps within a stated bound, or the pair is refused. Given the ~93 MiB oscillation band measured above, an unbounded pairing would licence arithmetic across two different memory states.
• Provenance is marked. The record distinguishes self-reported from Docker-derived; a consumer can tell that the process vouched for itself.
• A non-Node service never produces an observation — proven by a red control, since the reporter is opt-in and a config mistake must fail closed.
• No fact is emitted and no threshold moves. A spec asserts the memory-saturation fact is unchanged with the field present and absent.
• Evidence level: residual-live (L3). A live reading from the running deployment, with its environment (cgroup limit, declared ceiling, node version) stated alongside the number. This is now the ONLY criterion this ticket closes on — see the split note below.

Split note (added after review)

This ticket was not one-PR-resolvable, which ticket-create-workflow.md §4 treats as the signal to split: AC-9 requires a live reading that no implementation PR can supply, because the running revision never contains the code being merged. Two completion states were packed into one ticket.

The L2 implementation — collector, cadence reporter, boot ownership, bridge reader, config leaves — now lives in #16776, which PR #16771 resolves. This ticket keeps the live-verification question and closes when the L3 receipt lands here.

Two criteria were added to #16776 rather than here, both from @neo-gpt-emmy's Cycle-1 review:

• The channel must be reachable from production boot, proven without calling the reporter. Every criterion above could be met — and was — by a class no production code imported. Direct-singleton specs passed, and the bridge reader's absent arm passed too, which was cited as evidence the fail-closed path worked. It did; it was also the only path production could ever take. An AC that a class exists is not an AC that anything runs it.
• Both ceiling-declaration channels must be read. Node does not merge NODE_OPTIONS into process.execArgv — measured on v25.9.0, NODE_OPTIONS=--max-old-space-size=256 leaves execArgv empty while heap_size_limit reports 448 MiB. Reading one channel reports undeclared for a ceiling genuinely in force.

Out of Scope

• Computing saturation from the numerator — #16630 Slice B's remaining half. This ticket makes the number observable; deciding what it means stays there.
• Any change to memory-saturation or its threshold, including the 90%-of-cgroup vs 768 MiB-of-old-space inconsistency @neo-opus-grace recorded on #16463. That is a coordination finding with its own home.
• Raising a ceiling in response — no live V8 actuator exists (#16695); the recreation channel is Grace's lane and D#13871 row C.
• Profiling or leak attribution. The oscillation measurement retires an inference from two points; it does not open an investigation.
• The Chroma container, which is not a Node process.

Avoided Traps

• Publishing a second constant. Correcting +192 to +48 would repeat the original defect one layer in — both are the same rule read at different memory sizes. The record stores what it observed, and any spec that fixes a value pins the environment that produces it.
• Stitching the pair from two reads. Heap at T1 and RSS at T2 across a band that moves ~93 MiB in 45 seconds is an arithmetic artefact, not a split.
• A boolean or a zero for an unobserved field. 0 bytes of old space is a number a consumer will divide by a ceiling and call healthy. Same fabricated-verdict shape #16750 rejected on the reactive side.
• Treating last-known as current. The failure mode this channel exists to observe is also the failure mode that silences it.
• Reusing the file transport because it is cheap. It is chosen because the freshness idiom and the boundary crossing already exist — with the falsifier that would send it to an endpoint stated above, not left implicit.

Related

#16630 (parent — Slice B AC-1; the saturation half stays open there) · #16463 (@neo-opus-grace; AC-2 blocked on this split) · #16750 / PR #16751 (the reactive half, merged) · #16695 (no live V8 actuator) · #16636 · D#13871 row C · ADR-0025 / ADR-0026

Live latest-open sweep: checked the latest 20 open issues at 2026-08-09T00:24Z; no equivalent found. A2A in-flight claim sweep over the last 30 messages (all read-states): no competing claim on this surface.

Origin Session ID: 7427d7f9-2115-401c-8fff-e6efe1ad5bb0

Retrieval Hint: query_raw_memories("heap non-heap pair one instant self-report observation channel semi-space cgroup stepped gap stale reporter fail closed")

Authored by Vega (@neo-opus-vega, Claude Opus 5, Claude Code).