DeploymentStateBridge direct probes ship without credentials and always fail

Context

Since PR #16778 (2026-08-09) the Orchestrator's DeploymentStateBridge has run a direct-probe evidence channel that cannot succeed in this deployment — and every failure is logged as a warning pair roughly every 30 seconds. First observed 2026-08-23T20:19:59Z (five seconds after the current orchestrator container started); ~302 occurrences in its first 80 minutes. The channel's purpose is deployment-state evidence collection — the same surface the four-service runtime pin's acceptance used on 2026-08-23.

Live latest-open sweep: checked latest 20 open issues at 2026-08-24T01:0xZ; no equivalent ticket. A2A in-flight claim sweep (~60 min window, all read-states): no competing claim; @neo-gpt-emmy was offered this scope in A2A and has not claimed it.

The Problem

DeploymentStateBridgeService.collectDirectProbe() (ai/daemons/orchestrator/services/DeploymentStateBridgeService.mjs:471) calls runHealthcheck({url, clientName, identity, timeoutMs}) with no credential, against AiConfig.orchestrator.deploymentStateBridge.directProbeUrls = http://kb-server:3000, http://mc-server:3001. Both endpoints require auth. Every probe fails:

[DeploymentStateBridge] direct probe for kb-server produced no service evidence:
Streamable HTTP error: Error POSTing to endpoint: {"error":"invalid_token","error_description":"Missing Authorization header"}

The capability to fix it already exists: runHealthcheck supports two mutually exclusive credential carriers (NEO_MCP_HEALTHCHECK_TOKEN_FILE / --bearer-token-file, bearerTokenEnv option — ai/scripts/diagnostics/mcpHealthcheck.mjs:130-165), and the token secret exists on the host at ~/.neo-ai/secrets/mcp-auth-token (mounted into mc-server as a run secret today). What is missing is threading on both sides:

1. Service: collectDirectProbe never passes a credential option.
2. Compose/config: the orchestrator container mounts no secret material at all (docker.sock, state volumes, kb-config bind only), and no env var names a token source.

The Architectural Reality

• directProbeUrls is an AiConfig leaf (configBase.mjs, env NEO_DEPLOYMENT_STATE_BRIDGE_DIRECT_PROBE_URLS, csv) — config additions must follow the ADR-0019 leaf pattern.
• The kb/mc endpoints validate Bearer tokens; mcp-auth-token is the existing shared plane credential.
• The bridge classifies probe outcomes (classifyDirectProbeOutcome) — auth failures currently land in "no service evidence", which is why the channel fails silently rather than loudly naming its cause.
• Precedent for env-carried bearer credentials: ai/mcp/client/config.mjs bearerTokenEnvVar.

The Fix

Three surfaces, smallest first:

1. compose (ai/deploy/docker-compose.yml, orchestrator service): mount ~/.neo-ai/secrets/mcp-auth-token as /run/secrets/mcp-auth-token (same pattern mc-server already uses) and export NEO_DEPLOYMENT_STATE_BRIDGE_BEARER_TOKEN_FILE=/run/secrets/mcp-auth-token.

2. configBase.mjs: extend orchestrator.deploymentStateBridge with bearerTokenFile reading that env name via the standard leaf() pattern.

3. DeploymentStateBridgeService.mjs: thread it into runHealthcheck as the bearer-token-file carrier.

   Unset-case contract (clarified 2026-08-24 during review — the original text required two incompatible behaviors): when the leaf is UNSET, behavior is unchanged from today — no token-related warn; downstream sees the real invalid_token transport verdict, which is itself the honest signal that no credential was configured. The named-WARN requirement applies to the CONFIGURED-but-unreadable case (path + cause logged, contents never). Chosen because warning on every unset sweep would re-noise the very channel this fix silences.

Acceptance Criteria

• [L3-deferred — operator handoff needed] With the secret mounted and env wired, direct probes return service evidence for both kb-server and mc-server (live in-container receipt captured post-deploy of the merged head; unit arms prove everything short of the live plane)
• Without any credential configured, the warn line names the missing bearer-token file path rather than a generic transport error Amended 2026-08-24 (review round 2): the UNSET case produces NO token-related warn — silent unchanged legacy behavior, because warning on every unset sweep re-noises the channel this fix silences; the real invalid_token verdict downstream is itself the signal. The named-WARN requirement applies to the CONFIGURED-but-unreadable case only. Discharged by spec arm asserting zero token-related warns through an observed sink.
• Unit spec covers: credential threaded when leaf set; loud named error when leaf set but file unreadable; unchanged skip behavior when unset
• No other DeploymentStateBridge channel changes

Contract Ledger Matrix

Out of Scope

• Rethinking which services the bridge probes
• The ingress PAT model itself (operator-owned)
• Backfilling historical evidence lost to the blind window

Avoided Traps

• Silent default fallback: the module must not invent a default path that may not exist — missing config stays loud.
• Hardcoding the secret path in source: path arrives via config leaf; compose owns the mount.

Decision Record impact

aligned-with ADR-0019 (extends the existing leaf pattern; no new substrate).

Related

#16778 (shipped the uncredentialed channel) · #17621 (acceptance surface blinded by this) · #17646/#17647 (instrument-honesty family)

Origin Session ID: b644277f-7fcf-4079-a363-a7f9099a4566

Retrieval Hint: query_raw_memories("DeploymentStateBridge direct probe invalid_token Missing Authorization header bearer mcp-auth-token")