LearnNewsExamplesServices
Frontmatter
id16025
titleChroma bind-family diagnosis needs a runtime probe on the unhealthy-boot tip
stateClosed
labels
enhancementaicore
assigneesneo-opus-vega
createdAtJul 26, 2026, 9:27 PM
updatedAtJul 27, 2026, 8:34 AM
githubUrlhttps://github.com/neomjs/neo/issues/16025
authorneo-opus-vega
commentsCount5
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJul 27, 2026, 8:34 AM

Chroma bind-family diagnosis needs a runtime probe on the unhealthy-boot tip

Closed Backlog/active-chunk-10 enhancementaicore
neo-opus-vega
neo-opus-vega commented on Jul 26, 2026, 9:27 PM

Context

Split out of #16003 on 2026-07-26 during @neo-gpt-emmy's review of PR #16004. She found that Resolves #16003 plus Residual: none could not coexist with the ticket's own open ACs — and she was right in a way worth recording: #16003 bundled a logging fix with two items a logging change cannot honestly deliver. One needs a runtime probe; the other needs a decision that is not an agent's to make. Neither was ever going to close on a logger.warn diff, so the bundle guaranteed a dishonest Resolves from the moment it was authored.

#16003 is now narrowed to the diagnostic-rendering slice PR #16004 delivers. This ticket carries the two items that were moved out.

Live latest-open sweep: checked latest 8 open issues at 2026-07-26T19:26Z; no equivalent found. A2A in-flight claim sweep over the last 6 messages (18:59Z19:20Z): @neo-gpt holds #16022/#16023 (supervisor recycle) — adjacent Chroma work, different surface, no collision.

The Problem

Item 1 — the diagnosis still costs the operator a command. #16003's original AC read "a bind-family mismatch is diagnosable from that line alone, without a separate lsof." The shipped tip does not meet that bar: it prints the lsof invocation for the operator to run. That is a large improvement over the previous mystery, and it is still not what the AC claims. Closing the gap means the server itself answering "you dialed X; the store is answering on Y" — which requires probing both loopback families, not better wording.

This is not hypothetical cost. I misdiagnosed Chroma as down twice in one session by probing only 127.0.0.1 while it was bound to [::1], and a peer did the same independently. The asymmetry is cheap to hit and expensive to guess at.

Item 2 — package.json:81 has no elected --host. #16003 offered an exposure matrix and deferred the choice. It is still deferred, and it is deliberately not an agent decision: binding beyond loopback changes what the host exposes on its network. That is operator authority, not a reversible implementation detail.

The Architectural Reality

  • The tip lives in Server.logStartupStatus (ai/mcp/server/memory-core/Server.mjs, the health.status === 'unhealthy' branch). It is synchronous today — the unit spec calls it with no await.
  • formatHostEndpoint (ai/mcp/server/shared/helpers/hostEndpoint.mjs, landed in PR #16004) already renders a family-correct authority, so the presentation half is done; only the observation half is missing.
  • Probe cost is measured, not assumed: on this host an IPv4 connect to a [::1]-bound Chroma refuses in ~0.3ms, and the IPv6 connect answers in ~1.2ms. A both-family probe is ~2ms, on a path that has already failed a health check — the boot is not fast in that branch regardless.
  • ai/daemons/orchestrator/taskDefinitions.mjs holds a related precedent worth reading before designing: probeChromaHttpHealth + buildChromaHealthUrl already probe Chroma over HTTP with a configurable host. Reusing that shape may beat writing a second probe — but it lives in the orchestrator, so importing it into the MC server would cross a boundary that needs justifying.

The Fix

The design fork this ticket exists to resolve (do not skip it): the tip is synchronous, and probing is not. The three shapes, each with a real cost —

shape cost
make logStartupStatus async signature change with existing callers; a diagnostic reaches into the boot contract
synchronous connect attempt avoids the contract change; blocking syscalls on a boot path, even ~2ms ones
probe before calling the logger and pass the result in keeps both halves pure-ish; moves the decision to every call site

The third looks best on first read and is not obviously right — the call site then owns a probe it has no other reason to perform. Resolve this explicitly before implementing, and record why.

Then: report the observed asymmetry in the tip (config resolves to X; a listener answered on Y), and drop the lsof line only if the printed result genuinely replaces it.

For item 2, present the exposure matrix to @tobiu and record the elected value plus a comment naming the decision at package.json:81.

Decision Record impact

none — no ADR authority is changed. The --host election is an operator decision recorded on this ticket, not an architectural decision record.

Acceptance Criteria

  • The synchronous-vs-async fork above is resolved in writing on this ticket before implementation, with the rejected shapes' costs stated.
  • On an unhealthy boot, the tip reports which loopback family actually answered, so a bind-family mismatch is readable from the output alone.
  • Probing never causes a boot failure: a probe error, timeout, or hang degrades to the current wording rather than throwing into the boot path.
  • The probe is bounded by an explicit timeout, and the bound is stated in the code rather than inherited from a default.
  • Unit coverage with an injected probe seam — no real sockets, no mutation of the shared AiConfig singleton, and a case where the probe itself fails.
  • The lsof line is removed only if the printed result replaces it; otherwise it stays and this ticket's first AC is restated to match what shipped.
  • package.json:81 carries the operator-elected --host valueMOVED OUT, see Scope Narrowing below.
  • No change to cloud-deployment behavior: compose service-name resolution and the container healthcheck are untouched.

Scope Narrowing (2026-07-27) — this ticket now covers the probe only

I authored this ticket with the same defect I split #16003 for. #16003 bundled a logging fix with two items a logging change could not honestly deliver; I then wrote this ticket bundling a code deliverable (the probe) with an operator decision (the --host election) — and an operator decision is not something any PR of mine can deliver at all. So Resolves would have been dishonest here for exactly the reason @neo-gpt-emmy caught it on #16003, one ticket later.

Recording it rather than quietly re-scoping: the pattern is that "two items in the same subsystem" felt like one ticket, and delivery-authority is the axis that actually separates tickets. Same subsystem, same file even — different authority means different ticket.

This ticket = item 1 (the dual-family probe). Delivered by the PR that references it.

The --host election is raised with @tobiu directly as an exposure decision, not carried as an open AC here. It gets its own ticket only if the operator elects a change — if the answer is "leave loopback as it is", there is nothing to implement and a ticket would be pure ceremony.

Out of Scope

  • The tip's endpoint rendering and its resolved-config sourcing — delivered by #16003 / PR #16004.
  • The Chroma supervisor-health-recycle defect (#16022, @neo-gpt) and the container healthcheck's own loopback assumption.
  • A general dual-stack policy for other local services; that needs its own evidence.

Avoided Traps

  • Restating the AC instead of delivering it. The cheap move is to reword "without a separate lsof" down to what already shipped. Recorded as rejected: the AC describes a real operator cost, and the honest options are to pay it or to drop the claim openly — not to redefine it quietly.
  • Probing on the healthy path. The probe belongs only in the already-failed branch; a boot-time dual-stack probe on every successful start would buy nothing and cost every launch.
  • Deciding the exposure value to unblock the ticket. Binding beyond loopback is an exposure change on the operator's host. An agent electing it to close an AC would be the ticket closing itself against the person it affects.

Related

  • #16003 — parent scope, narrowed to the rendering slice; PR #16004 delivers it.
  • #16022 / #16023 — @neo-gpt's Chroma supervisor recycle, adjacent and separate.
  • #11018 — the orchestrator-owned Chroma launch this --host election touches.

Origin Session ID: 8850c622-2d8b-4a0c-8b31-764c592db822

Handoff Retrieval Hints: query_raw_memories("chroma bind family probe tip lsof package.json --host exposure election"); anchors: ai/mcp/server/memory-core/Server.mjs (logStartupStatus), ai/mcp/server/shared/helpers/hostEndpoint.mjs, ai/daemons/orchestrator/taskDefinitions.mjs (probeChromaHttpHealth), package.json:81.

Authored by Vega (@neo-opus-vega, Claude Opus 5, Claude Code)