LearnNewsExamplesServices
Frontmatter
id16324
titleFleet server mailbox bindings ride the containerized MC plane
stateClosed
labels
enhancementaiarchitecture
assigneesneo-fable-clio
createdAtAug 2, 2026, 3:09 AM
updatedAtAug 2, 2026, 1:58 PM
githubUrlhttps://github.com/neomjs/neo/issues/16324
authorneo-fable-clio
commentsCount0
parentIssue13015
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 2, 2026, 1:58 PM

Fleet server mailbox bindings ride the containerized MC plane

Closed Backlog/active-chunk-11 enhancementaiarchitecture
neo-fable-clio
neo-fable-clio commented on Aug 2, 2026, 3:09 AM

Context

Lane A leaf 1 of the FM week lane map (#14560, comment IC_kwDODSospM8AAAABMzhuWA; operator direction 2026-08-02: FM connects to the dockerized Agent OS with real data, webhook-class transport, cloud-parity).

The 2026-08 hard cut (#16167, D#15758 cohort) moved Memory Core and Knowledge Base into containers (neo-local-agent-os-mc-server-1, plane neo-local-canonical, dataRoot /app/.neo-ai-data, ingress at 127.0.0.1:3102). The Fleet server predates the cut: it binds Memory Core in-process and therefore reads/writes a host-side plane that is no longer the canonical truth. Every cockpit A2A surface (activity stream, per-agent mailbox mirror, operator compose, catch-up) renders the wrong plane — the operator sees honest-degrade/sample states or stale host data while the real fleet coordinates through the container.

Live latest-open sweep: checked latest 20 open issues at 2026-08-02T01:0xZ — no equivalent (closest: #16310 wake arming, #16320 schema pinning, #16206/#16208 compose layering; all different concerns). A2A in-flight claim sweep (status:all, limit 30, ~60min window): no overlapping claim; Lane A claimed by this author in the #14560 co-plan.

Body updated 2026-08-02 (pre-implementation, by the author): full entry read surfaced a FOURTH in-process MC seam (the catch-up source) — same class, folded in; and the single-viewer identity invariant is now explicit (boot-time plane admission must prove the bearer subject IS the boot-resolved viewer). No scope change beyond the same seam class at the same entry.

Body updated 2026-08-02 (post-review fold, by the author, per PR #16329 cycles 1-2): the shipped client rides the official MCP SDK Client + StreamableHTTPClientTransport (not the drafted hand-rolled handshake); the error-containment wording is corrected below (the client THROWS like the in-process services; the ADAPTERS own degraded/denied mapping); plane-mode viewer boot is GRAPHLESS (resolveFleetViewerClaim + plane-side proof — a stale host graph cannot veto a healthy plane); recovery is single-flight with replay gated to proven session-invalidity (no ambiguous mutating replay); AC states below reflect delivered evidence.

The Problem

devFleetServer.mjs binds the mailbox + history surfaces by importing the memory-core singletons into its own process:

  • devFleetServer.mjs:95-101import('../memory-core/MailboxService.mjs')listMessages: MailboxService.listMessages.bind(MailboxService) handed to wireFleetActivityReadSource (the A2A activity slot).
  • devFleetServer.mjs:110addMessage: MailboxService.addMessage.bind(MailboxService) (the #15376 operator compose verb).
  • devFleetServer.mjs:144-150 — the #15269 mailbox mirror binds mailboxService: MailboxService the same way.
  • devFleetServer.mjs:130-140 — the S3 catch-up source calls the in-process memory-core toolService.callTool('explore_memory_history'|'explore_pull_request_history', args) directly.

In-process binding was correct pre-cut (one plane, one process family). Post-cut it is a split-brain read: the host process resolves a host data root while the canonical plane lives in the container volume — exactly the divergence class Sol's 2026-08-01 audit flagged (session 1fb83843, "split-write risks between host and Docker volumes") and Emmy's #15931 tracks for data-root re-derivation. D#16304 (merged code does not reach running containers) makes the two planes drift further with every merge.

The Architectural Reality

  • The adapters are already DI-clean and do not change: fleetMailboxMirrorAdapter.mjs ("consumes an injected MailboxService-compatible listMessages() read path"; admission = the service's own CAN_READ_INBOX_OF, never re-implemented), fleetA2AActivityAdapter.mjs (injected listMessages, never imports the singleton), wireFleetActivityReadSource.mjs ("the caller binds… identity/permission binding stays at the boundary"), wireFleetCatchUpSource.mjs (injected operation callers). The seams were designed for exactly this swap.
  • The containerized MC serves streamable-HTTP MCP at <base>/mc/mcp with bearer admission — the same surface every managed seat consumes (RunningTheFleetCockpit.md; #14574 connectTenant derives + verifies it fail-closed). list_messages / add_message / explore_memory_history / explore_pull_request_history are served tools on that plane; RLS scopes them by the server-resolved bearer subject.
  • The single-viewer identity invariant (the crux): in-process, admission runs under the per-request viewer RequestContextService carries; via a plane bearer, every call runs as the bearer's server-resolved subject. devFleetServer is single-viewer by design today (viewer resolved at boot; D#16176 falsifier 1). Plane mode is therefore only honest when the bearer's plane-side subject IS the boot-resolved viewer — the tenant machinery already ships the exact verification (FleetTenantService.mjs:678 probeMcpIdentity with expectedIdentity, riding list_permissions which "returns the canonical identity it actually used"). A bearer resolving to anyone else is an identity swap and must refuse startup in plane mode, fail-closed with a named reason.
  • Config: per ADR 0019, plane base + bearer are AiConfig leaves in the existing fleet subtree (ai/configBase.mjs:229), read at the boot use site. Default plane base is EMPTY = in-process binding (correct for container-less contributor machines); the canonical dockerized machine activates plane mode via its operator overlay / env delta (NEO_FLEET_PLANE_BASE=http://127.0.0.1:3102). No module-level process.env reads (antipattern A1), no pass-along config threading (Group B), no re-derived data roots (#15931 class).
  • Wake-dispatch coupling is server-side plane behavior and is untouched here (the busy-seat hang class from 2026-08-02 belongs to the #16246/#16253 degrade thread).

The Fix

One new module + one entry swap; adapters and admission logic untouched:

  1. ai/services/fleet/planeMailboxClient.mjs (sibling-lift beside createBootIdentityReadSource.mjs / readActiveWakeSubscriptionIdentities.mjs; structure-map gate run 2026-08-02, ai/services/fleet = 40 files, owning folder confirmed) — as-shipped contract (folded from the PR #16329 review cycles): the official MCP SDK Client + StreamableHTTPClientTransport (request-id uniqueness and response correlation SDK-owned) against the configured plane base, endpoint-validated through the shared normalizeSecureMcpEndpoint policy before any request. init({expectedIdentity}) proves the bearer's plane-side subject via list_permissions and refuses fail-closed on mismatch; the SAME proof gates every re-establishment. The client THROWS like the in-process services (transport failures bounded; tool isError text passed through) — the ADAPTERS own degraded/denied mapping. Lifecycle: single-flight, generation-safe proven acquisition (concurrent recoverers share one proof; an existing replacement is ridden, never overwritten); replay only on proven session-invalidity (the SDK 404 class) — ambiguity throws for every tool, so mutating add_message can never double-send; close() is a shared terminal barrier (fences in-flight establishment, one promise for all closers, epoch reset on re-init). Named wrappers listMessages(args) / addMessage(args) keep MailboxService-compatible shapes.
  2. devFleetServer.mjs entry swap: resolve the plane leaves at boot; when the plane base is configured, init the client (fail-closed on identity mismatch) and bind the FOUR seams (:101 listMessages, :110 addMessage, :137-139 catch-up operations, :150 mirror) to it; when the base is empty, keep the current in-process binding as explicit logged fallback (dev-loop without containers stays functional). The wire modules' fail-soft/unanimity semantics are preserved unchanged.
  3. AiConfig leaves in the fleet subtree: planeBase (NEO_FLEET_PLANE_BASE, default '') + planeBearer (NEO_FLEET_PLANE_BEARER, default ''); declared once, resolved by the leaf machinery.

Contract Ledger

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
Injected listMessages/addMessage (3 seams, devFleetServer.mjs:101/:110/:150) This ticket Plane-client-bound when fleet.planeBase resolves In-process binding (explicit, logged) JSDoc at entry Unit + live receipt
Catch-up operation callers (devFleetServer.mjs:137-139) This ticket (name, args) => client.callTool(name, args) when plane-bound In-process toolService.callTool JSDoc at entry Unit + live receipt
planeMailboxClient.mjs This ticket Official SDK client/transport (ids + correlation SDK-owned); THROWS like the in-process services (adapters own degraded/denied mapping); identity-proven at every establishment; single-flight recovery; replay only on proven session-invalid (404) n/a (new) Module JSDoc Unit tests + headed receipt
fleet.planeBase / fleet.planeBearer leaves ADR 0019 (AiConfig SSOT) leaf('', ENV, 'string') in the fleet subtree Empty = in-process mode Leaf JSDoc Existing leaf conventions
Adapter surfaces (mirror/activity/catch-up) #15269/#14562/S3 (unchanged) No change n/a n/a Existing specs stay green

Decision Record impact

aligned-with ADR 0019 (leaves at entry, no re-derivation); aligned-with the #16167 hard-cut direction. Does NOT touch D#16176's diamonds: Fleet control-plane placement, request-scoped multi-viewer identity, host actuation, and owner-scoped stores stay exactly where they are — this ticket only changes which plane the existing single-viewer host process reads, via the shipped seat-consumption pattern (#14574 class), with the single-viewer invariant made mechanical (bearer subject === boot viewer, verified at init). No new authority model.

Acceptance Criteria

  • planeMailboxClient.mjs ships with unit tests — delivered beyond the drafted list: 28 specs against the REAL SDK stack (endpoint boundary, handshake + identity proof, error shapes incl. the admission-text contract, production-shape concurrency, replay boundary with the commit-once mutating witness, single-flight no-orphan recovery, changed-identity rejection, bricked-client regression, awaited idempotent close, plus the four lifecycle witnesses: dual-404 + slow-teardown generation safety, close-during-establishment fencing, close-during-failure-teardown gated-DELETE, concurrent-closer single-teardown).
  • devFleetServer binds all FOUR seams to the plane client when fleet.planeBase resolves non-empty AND init's identity verification passes (witnessed: named 401 refusal + exit 1 on a bad bearer; plane-verified boot line; explicit in-process fallback line). Plane-mode viewer boot is graphless per the post-review fold.
  • Config is ADR-0019-clean: leaves declared once in the fleet subtree, read at the entry use site; zero process.env reads in the new module; SSOT lint green (it caught and killed one hidden-default on the first draft).
  • Honest-degrade preserved — witnessed LIVE twice: a slow plane produced named timeout degrades (cycle 1), and the WAL-dead plane window produced the precise "ambiguous — not replayed" reason end-to-end in the pane (cycle 2); no fabricated data, no crash either time.
  • Live receipt against the running local plane: headed cockpit at the public receipt tree (receipt/16324-headed-receipt-tree) rendered the roster live, ActivityStream live/streaming, and the operator mailbox pane with REAL current traffic (admission: granted, viewer-stamped, paging); operator compose round-tripped with an independent MCP-client read-back. Current-head re-run receipted boot proof + roster live + the honest-degrade chain (the plane's known degraded window); full-rows-at-merged-head listed in PR Post-Merge Validation.

Out of Scope

  • Fleet control-plane containerization, multi-viewer request identity, host actuator — D#16176 (S1 track).
  • fleetWakeStateAdapter / throttle-state plane migration — separate Lane A leaf (filesystem-shaped, not tool-shaped).
  • PR/lane activity slot (issuesDir/pullsDir local-synced reads) — git-synced content, correctly local, unchanged.
  • Boot-identity read source (:71, orchestrator runtime-state dir) — orchestrator observability belongs to Lane C.
  • Tenant-registration cockpit UX (connectTenant UI as default-plane picker) — follow-up leaf.
  • KB-plane consumption by FM — nothing in FM reads KB today.

Avoided Traps

  • Mounting the container volume / reading shared SQLite from the host — D#16176 Diamond-1 Option B's named split-brain trap; rejected without waiting for that round: the plane's API is the only supported read surface.
  • Per-request or per-source dataDir mutation — D#16176 settled constraint, violates ADR 0019.
  • A second truth cache (replicating mailbox rows into fleet stores) — Diamond-1 Option C's consistency surface with zero measured need.
  • Silent dual-plane mode (plane for some seams, in-process for others, unlogged) — the binding decision is all-or-nothing per boot and logged, or the operator cannot trust what the cockpit shows.
  • Silent viewer swap — plane mode with a bearer resolving to a different subject than the boot viewer would re-attribute every admission decision; fail-closed refusal instead.

Related

Parent: #13015 (FM MVP Brain spine — the S1 mirror #15269 lives there). Lane map: #14560. Machinery precedent: #14574. Compose verb: #15376. Data-root leak class: #15931. Hard cut: #16167, D#15758, D#16304. Gated sibling track: #16168 / D#16176.

Origin Session ID: 96ee8bfe-9dd2-4fd9-9532-304df7044dc6

Retrieval Hint: "fleet plane client mailbox bindings containerized MC" · query_summaries("FM week lane map spine reconnect")

tobiu referenced in commit a0e6c6f - "feat: fleet mailbox seams ride the containerized MC plane (#16324) (#16329) on Aug 2, 2026, 1:58 PM
tobiu closed this issue on Aug 2, 2026, 1:58 PM