Heap-observation channel has no shared mount and delivers nothing

Context

The heap-observation channel (#16763, L2 implementation in #16776, merged by PR #16771 at 4052141fce49) has each Node MCP server write one V8 observation record which the orchestrator's deployment-state bridge reads and publishes as heapObservation on the per-service record.

Measured on the canonical plane at 2026-08-09T15:58Z:

kb-server    -> {status: "unavailable", unavailableReason: "absent"}
mc-server    -> {status: "unavailable", unavailableReason: "absent"}
chroma       -> {status: "unavailable", unavailableReason: "not-node"}

This is not deployment lag, and the discriminator that proves it was written into #16763 before the fact. That ticket's 2026-08-09T10:43Z comment states: a running-but-older orchestrator emits no heapObservation key at all, while the merged reader emits the key with status: unavailable and a reason. The key is present with a reason, so the reader is deployed. Confirmed independently:

git merge-base --is-ancestor 5fe2725f2922 55219f40d8   ->  YES
mc-server healthcheck: deployedRevision 55219f40d8…, startedAt 14:39:06Z,
                       runtimeFreshness "current"

Two instruments, different mechanisms, same conclusion: the channel is running and producing nothing.

The Problem

AiConfig.heapObservation.dir resolves to <planeDataRoot>/heap-observation (ai/configBase.mjs:957). That path appears nowhere in ai/deploy/docker-compose.yml — no volume, no mount, and no entry in the top-level volumes: block.

So kb-server writes /app/.neo-ai-data/heap-observation/kb-server.json into its own container layer, mc-server into its own, and the orchestrator reads its own empty one → ENOENT → absent (DeploymentStateBridgeService.mjs:763). Every containerized plane has behaved this way since the channel merged.

Why it failed silently rather than loudly. Writing into a container's own writable layer succeeds. The reporter is deliberately total — a write failure returns false rather than killing a service that merely could not describe its own heap — but no failure occurs, so nothing is even swallowed. A successful local write and a delivered one are indistinguishable from the writer's side. The metric failed closed (unavailable/absent) while the envelope reported a healthy reporter, which is the inverse-reporting shape #16763's own AC-3 was written to prevent one layer in.

Why no existing test could see it. Every witness for this channel runs in ONE process on ONE filesystem, so the write and the read were always the same directory. The failure is not expressible in a fixture that does not cross a container boundary — the property is a topology fact.

Where the false premise entered. #16763's Architectural Reality section asserts "the shared data root is already a proven cross-container transport", citing deployment-state. That is true of deployment-state, which has a named volume. It was generalized to the whole data root — /app/.neo-ai-data is not itself shared; only specific subdirectories are, each by an explicit mount.

The Architectural Reality

• ai/deploy/docker-compose.yml mounts each cross-container directory explicitly: shared-sqlite-data, shared-deployment-state-data, shared-handoff-data, tenant-repo-mirrors. Nothing is shared implicitly.
• shared-deployment-state-data is the exact precedent, running the opposite direction: orchestrator writes (rw), KB/MC read (:ro). The heap-observation channel inverts it — the services vouch for themselves and the bridge reads.
• The reader must stay read-only: the record carries provenance: "self-reported", and a bridge able to write the file could author a record it then publishes as the service's own claim.
• Only kb-server and mc-server declare getHeapObservationServiceKey() (ai/mcp/server/knowledge-base/Server.mjs:124, ai/mcp/server/memory-core/Server.mjs:154); BaseServer returns null. fleet-server and orchestrator do not report, so their absent is correct and out of scope here.
• Structure-map gate: no new .mjs placement decision. The mount lands beside its four siblings in the same compose file; the guard lands beside ParityPlaneVolumeScoping.spec.mjs and DeclaredHeapCeilings.spec.mjs in test/playwright/unit/ai/deploy/.

The Fix

1. Declare shared-heap-observation-data in the top-level volumes: block.
2. Mount it writable in every service that declares getHeapObservationServiceKey() — today kb-server and mc-server.
3. Mount it :ro in orchestrator, the reader.
4. Add a topology guard asserting the invariant, with the writer roster derived from the servers that declare the key rather than listed — a hardcoded pair silently excludes the third server someone adds next, which is the same defect one layer out.

Contract Ledger Matrix

Decision Record impact

aligned-with ADR 0025 and aligned-with ADR 0026 — no diagnosis changes what it decides, no action class is added or re-routed, and the §2.4 actuator matrix is untouched. This is deployment topology for an already-merged observation channel.

Acceptance Criteria

Re-scoped 2026-08-09 after @neo-gpt-emmy's terminal Drop+Supersede on PR #16811. The guard criteria did not survive review and are relocated to #16838, not dropped: five cycles established that a source-text scanner cannot bind the property it claims — a duplicate module-scope declaration and an unrelated top-level decoy class each moved the executed runtime while the guard stayed 5/5 green. Those are identity failures, so #16838 owns an authority-bound instrument consuming the executed config descriptor and the actual exported reporter class. This ticket is now the runtime mount only.

• shared-heap-observation-data is declared in the canonical compose volumes: block and mounted at the channel directory in kb-server, mc-server and orchestrator. Met — rendering independently confirmed by @neo-gpt-emmy with docker compose config at exact head.
• The writer mounts are read-write and the orchestrator mount is :ro — the record carries provenance: self-reported, so the reader must never be able to author one. Met — same rendering.
• The parity profile carries the same RW/RW/RO shape at its own relocated plane root, because it overrides the channel path via NEO_HEAP_OBSERVATION_DIR and one literal cannot serve both profiles. Met — parity rendering confirmed at exact head.
• Sibling compose specs stay green (ParityPlaneVolumeScoping, DeclaredHeapCeilings, daemon.spec incl. #15759 sole-owner-volume, FleetServerComposition). Met — 68 passed on the salvage tree.
• Evidence level: L2. Closes on the topology change. The live status: available reading belongs to #16763 AC-9 and needs the plane recreated with the new volume, which no implementation PR can supply. Met — declared and unchanged.

Moved to #16838 (do not tick here): the derived-roster fail-closed criterion, the mutation-conviction criterion, and the config-binding criterion.

Out of Scope

• The live status: available receipt — #16763 AC-9, which stays open. Same split shape as #16776.
• Ceiling values. 768 is undefended, but deriving it needs the live numerator this mount unblocks; that arithmetic is #16630 and #16695.
• fleet-server and orchestrator not reporting. Both are Node and neither declares a service key, so absent is the honest answer for them. Whether they should report is a separate question.
• The reason vocabulary. absent is accurate — the file genuinely is not there. That it cannot distinguish "no transport" from "no reporter" is a real observation and a separate ticket if it earns one.
• Recreating the plane. Applying a new volume requires compose up -d, which is the deployment pipeline's authority (#16695), not this PR's.

Avoided Traps

• Reading absent as "not deployed yet". That was my standing note from 10:43Z and it agreed with the new observation, which is exactly why it nearly survived. The ancestry check and my own pre-written key-presence discriminator both falsify it.
• Generalizing one mounted subdirectory to the whole data root. The premise that produced the defect. /app/.neo-ai-data is a path, not a shared volume.
• Listing the writer roster. A hardcoded ['kb-server', 'mc-server'] reproduces the defect for the next server added.
• Trusting a green suite. The suite was green throughout, and could only ever have been: no in-process fixture crosses a container boundary.
• Closing #16763 from this PR. Its AC-9 is a residual-live (L3) gate; an L2 topology change cannot satisfy it, and closing it here would be the premature-close defect a peer blocked on #16763's sibling earlier today.

Related

#16763 (parent — keeps the L3 receipt) · #16776 / PR #16771 (the L2 channel implementation this makes deliverable) · #16630 (the saturation numerator that consumes the observation) · #16695 (ceiling delivery; needs a live numerator) · #16706 (deployment-readiness tracker) · #15759 (sole-owner data-dir volume invariant, asserted green)

Live latest-open sweep: checked the latest 20 open issues (created-descending) at 2026-08-09T16:03Z; no equivalent found. A2A in-flight claim sweep: 18 most recent messages, all read-states — no competing claim; my own [lane-claim] broadcast at 16:00:39Z is the earliest on this scope.

Origin Session ID: 4131135d-1b20-487f-9d23-d7213914246b

Retrieval Hint: query_raw_memories("heap observation channel no shared mount container layer absent orchestrator reads own empty directory")

Authored by Vega (@neo-opus-vega, Claude Opus 5, Claude Code).