LearnNewsExamplesServices
Frontmatter
id12765
titleExpose MCP runtime source/schema drift in healthcheck
stateClosed
labels
enhancementdeveloper-experienceaiarchitecturemodel-experience
assigneesneo-gpt
createdAtJun 8, 2026, 3:53 PM
updatedAtJun 8, 2026, 4:36 PM
githubUrlhttps://github.com/neomjs/neo/issues/12765
authorneo-gpt
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 8, 2026, 4:36 PM

Expose MCP runtime source/schema drift in healthcheck

Closed v13.0.0/archive-v13-0-0-chunk-17 enhancementdeveloper-experienceaiarchitecturemodel-experience
neo-gpt
neo-gpt commented on Jun 8, 2026, 3:53 PM

Context

During the v13 cost-safety lane on 2026-06-08, the team hit an avoidable ghost-debugging loop around list_issues: the active GitHub Workflow MCP tool exposed to this Codex harness did not offer a compact title-only projection, so the tool returned noisy full issue bodies during backlog scans.

Verify-before-assert changed the diagnosis. Current source already contains the feature:

  • ai/mcp/server/github-workflow/openapi.yaml declares list_issues projection with full, summary, title, and title_only.
  • ai/services/github-workflow/IssueService.mjs implements projection='full'|'summary'|'title'|'title_only' and normalizes title_only to title.
  • test/playwright/unit/ai/services/github-workflow/IssueService.spec.mjs covers the projections; npm run test-unit -- test/playwright/unit/ai/services/github-workflow/IssueService.spec.mjs passed 40/40 on 2026-06-08.
  • The active github-workflow.healthcheck() still reported only status, timestamp, package version, GitHub CLI status, and notification preview. It did not expose the server process boot SHA, current repo SHA, OpenAPI/tool-list digest, process start time, or any stale-runtime warning.

So the real friction is not a missing list_issues feature. It is that long-lived MCP attachments can keep stale tool schemas after source changes/merges, while healthcheck still reads healthy and gives agents no cheap way to distinguish "feature missing" from "attached MCP process stale; restart/re-handshake needed".

Live latest-open sweep: checked latest 40 open issues at 2026-06-08T13:52:41Z; no equivalent live runtime freshness / source-schema drift healthcheck ticket found. Targeted open search for MCP stale tool shape healthcheck found no equivalent.

Exact/local sweep: rg "stale MCP|MCP.*stale|tool schema.*stale|schema drift|listTools.*drift|restart.*MCP|MCP.*restart|tool metadata" resources/content/issues resources/content/archive/issues resources/content/discussions ai .agents/skills found related but non-equivalent predecessors: #10248 documents fresh MCP client cache-bypass strategy, #11687 adds source-side cross-server listTools smoke coverage, and several restart/cache tickets cover Memory Core wake/message caches. None adds live healthcheck/source-freshness metadata for an already-attached MCP process.

Semantic sweep: query_documents(type='ticket', query='stale MCP server schema tool metadata restart drift') surfaced older schema/cache issues; inspected closest local hits #9837, #10248, and #11687, which are adjacent but not this runtime-drift indicator.

The Problem

Agents cannot cheaply answer: "is the MCP server I am currently attached to running the repo/tool schema I am looking at?"

That creates a recurring failure mode:

  • Source and tests prove a tool shape exists.
  • The active MCP connection still exposes an older tool schema or behavior.
  • healthcheck says healthy, because infrastructure is alive.
  • The agent misdiagnoses the gap as missing functionality and may file duplicate tickets or patch already-delivered source.
  • The actual fix is a harness/MCP restart or fresh-client re-handshake, but the runtime gives no structured freshness signal.

This is different from source-side schema drift. Source-side drift is already guarded by #11687. The gap here is runtime freshness: the process can be healthy and internally consistent while still stale relative to the checkout or merged dev state the agent is inspecting.

The Architectural Reality

  • MCP clients cache tool definitions at connection/handshake time; .agents/skills/debugging-antigravity/references/debugging-guide.md documents the stale-cache/fresh-client strategy.
  • Each Neo MCP server already has a healthcheck surface, but current outputs do not expose source freshness metadata.
  • ai/mcp/server/*/openapi.yaml and per-server toolService.mjs define the tool schema that long-lived clients cache.
  • #11687 verifies that current source can list tools correctly; it does not prove that an already-attached harness has reloaded those definitions.
  • A runtime comparison between boot-time source identity and current checkout identity can be cheap and read-only.

The Fix

Add a lightweight runtime freshness section to MCP healthchecks, starting with servers whose tool shape changes frequently (github-workflow, memory-core, knowledge-base) and using a shared helper if the pattern is reusable.

Suggested shape:

  • Capture at server boot:
    • package/version already reported,
    • startedAt,
    • boot git commit SHA when the server started,
    • boot OpenAPI digest and/or boot tool-list digest.
  • On healthcheck:
    • read current repo HEAD SHA when available,
    • recompute current OpenAPI/tool-list digest when cheap,
    • return a runtimeFreshness object with status: 'current'|'stale'|'unknown', boot/current SHAs, digest comparison, and a human-readable restart hint.
  • Treat drift as a warning/degraded subfield, not necessarily a full server outage. A stale but functioning MCP process can still be usable for read-only work; the point is to prevent false source claims.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
MCP healthcheck response Each server's healthcheck service + openapi.yaml response schema Include runtimeFreshness with boot/current source identity and stale-runtime warning status:'unknown' when git metadata or digest cannot be read OpenAPI schema + healthcheck JSDoc Unit tests with injected boot/current SHA and digest readers
Tool-shape freshness toolService.listTools() + server openapi.yaml Detect when the running process/tool-list digest differs from current checkout source Report warning without rejecting normal tool calls Healthcheck details/restart hint Fixture test simulating digest mismatch
Agent operational workflow Debugging guide / self-repair references Agents can use healthcheck to decide restart vs source patch Fresh MCP client strategy remains the manual fallback Short doc pointer if needed Regression note in PR body

Decision Record impact

none. This is healthcheck observability for existing MCP runtime behavior and aligns with the stale-cache documentation from #10248 and source-side listTools coverage from #11687.

Acceptance Criteria

  • At least github-workflow.healthcheck() exposes a runtimeFreshness field with boot source identity and current checkout identity when git metadata is available.
  • The field distinguishes current, stale, and unknown states without collapsing stale-runtime into a generic healthy/unhealthy boolean.
  • A stale boot SHA or stale OpenAPI/tool-list digest produces a clear restart/re-handshake hint in healthcheck details.
  • The healthcheck OpenAPI schema is updated for the new field so strict MCP clients do not reject the response.
  • Unit coverage proves the current/stale/unknown classifications via injected SHA/digest readers without requiring real GitHub credentials or live MCP harness restarts.
  • The PR body documents how this differs from #10248 fresh-client guidance and #11687 source-side listTools smoke coverage.

Out of Scope

  • Automatically restarting MCP servers or IDE harnesses.
  • Replacing the fresh MCP client cache-bypass strategy from #10248.
  • Reworking MCP client handshake semantics.
  • Changing the list_issues projection feature; that is already present in current source.
  • Treating every source mismatch as a hard outage. Some stale processes are safe enough for read-only work when the agent knows they are stale.

Avoided Traps

  • Do not file another title-only issue-list ticket; V-B-A proved that feature already exists in source.
  • Do not rely on package version alone; v13 development can change tool schemas many times without a package version bump.
  • Do not make this a CI-only guard; CI cannot tell a running agent whether its already-attached MCP process is stale.
  • Do not hide stale-runtime under status:'healthy' with no structured detail; that is exactly the false-confidence failure mode.

Related

  • #10248 — closed; documents fresh MCP client cache-bypass strategy.
  • #11687 — closed; source-side cross-server listTools smoke coverage.
  • #12763 / PR #12764 — current-session source-vs-runtime freshness pressure from provider/tool-shape work.
  • #12740 — v13 cost-safety epic context where stale MCP tool shape nearly produced duplicate work.

Origin Session ID: e8f07ef9-ef7e-4815-8ff4-7abe13720621

Handoff Retrieval Hints: query_raw_memories("stale MCP tool schema projection healthcheck runtimeFreshness"); exact source anchors ai/mcp/server/github-workflow/openapi.yaml list_issues projection, ai/services/github-workflow/IssueService.mjs listIssues projection, .agents/skills/debugging-antigravity/references/debugging-guide.md Fresh MCP Client Isolation Strategy.

tobiu referenced in commit dc31bd1 - "fix(ai): expose mcp runtime freshness (#12765) (#12766)" on Jun 8, 2026, 4:36 PM
tobiu closed this issue on Jun 8, 2026, 4:36 PM