LearnNewsExamplesServices
Frontmatter
id16685
titlePersist unfinished Memory Core tool-call telemetry
stateClosed
labels
enhancementaiperformanceagent-os
assigneesneo-gpt-emmy
createdAtAug 8, 2026, 2:16 PM
updatedAtAug 8, 2026, 3:51 PM
githubUrlhttps://github.com/neomjs/neo/issues/16685
authorneo-gpt-emmy
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 8, 2026, 3:51 PM

Persist unfinished Memory Core tool-call telemetry

Closed Backlog/active-chunk-13 enhancementaiperformanceagent-os
neo-gpt-emmy
neo-gpt-emmy commented on Aug 8, 2026, 2:16 PM

Context

The live incident in #16677 showed that Memory Core can stop completing MCP calls while its server process remains alive. Existing tool-call telemetry is written only after a call returns, so the operation owning a process wedge is absent from the diagnostic record precisely when it is needed.

The current recorder already persists redacted completed-call metadata and get_memory_core_tool_metrics already exposes aggregate counts, failures, and latency. A Knowledge Base query for unfinished active-call telemetry found that completed-call precedent but no start-boundary mechanism.

The Problem

Completion-only telemetry cannot distinguish “no request reached dispatch” from “request entered dispatch and never completed.” A future recurrence therefore still lacks a durable operation identity after the process recovers or restarts.

The recorder must remain strictly best-effort. Diagnostics must not add an unbounded SQLite wait to the MCP path, store raw arguments/results, or make the metrics observer report itself as an active problem.

The Architectural Reality

The Agent OS structure map confirms the existing ownership split:

  • ai/services/memory-core/MemoryCoreRecorderService.mjs owns the redacted SQLite telemetry row and aggregate query.
  • ai/mcp/server/memory-core/toolService.mjs owns the transport-visible dispatch boundary.
  • ai/mcp/server/memory-core/openapi.yaml owns the public diagnostic response contract.
  • test/playwright/unit/ai/services/memory-core/MemoryCoreRecorderService.spec.mjs owns the recorder and facade regression coverage.

No new file or service boundary is required.

The Fix

  1. Persist a redacted row immediately before non-observer tool dispatch and return an opaque call ID.
  2. Complete that same row in the existing finally boundary.
  3. Add an idempotent completed_at schema migration that preserves legacy completed rows.
  4. Separate completed aggregates from oldest-first unfinished rows in get_memory_core_tool_metrics.
  5. Expose only tool name, opaque call ID, start time, and elapsed time for unfinished rows.
  6. Bound telemetry SQLite lock waiting and preserve fail-open dispatch behavior.
  7. Keep get_memory_core_tool_metrics out of its own unfinished snapshot while retaining its completed latency row.

Contract Ledger

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
Tool dispatch start boundary toolService.mjs Record a redacted start row before non-observer dispatch A recorder failure returns null and dispatch continues Inline JSDoc/comment Facade unit test
Telemetry row lifecycle MemoryCoreRecorderService.mjs Insert at start; update the same row at completion Completion without a start ID inserts the existing completed form Method JSDoc SQLite row-count and migration tests
Diagnostic response Memory Core OpenAPI Return completed aggregates plus bounded unfinished rows Disabled/unavailable states return empty collections OpenAPI description/schema OpenAPI compliance and recorder tests
Sensitive payload boundary Recorder redaction contract Store byte counts, never raw arguments/results; omit agent/session identity from unfinished output Fail open without telemetry OpenAPI + method JSDoc Secret-string negative assertions

Decision Record impact

None. This extends the existing Memory Core telemetry and MCP ownership boundaries without adding a diagnosis/action class or changing ADR 0025/0026 recovery authority.

Acceptance Criteria

  • A tool that crosses the dispatch start boundary has a durable row with an opaque ID, tool name, start timestamp, and no completion timestamp.
  • The completion boundary updates that same row rather than creating a duplicate.
  • Legacy completed rows migrate idempotently and are never reclassified as unfinished.
  • Completed aggregates exclude unfinished rows and expose a separate total plus oldest-first bounded unfinished list.
  • Unfinished output contains no raw arguments/results and no agent/session identifiers.
  • The metrics observer does not report itself as unfinished but retains completed latency telemetry.
  • SQLite contention on the telemetry path is bounded and recorder failure cannot block tool dispatch.
  • Focused recorder and OpenAPI contract coverage passes under Neo's unit Playwright configuration.

Out of Scope

  • Identifying or repairing the CPU/event-loop mechanism behind #16677.
  • Claiming that an unfinished row is necessarily still active; it may be abandoned by a prior process.
  • Adding automatic restart, replay, restore, or retry behavior.
  • Adding event-loop, heap/GC, pulse, or container-health instrumentation beyond the existing tool recorder.
  • Closing #16677.

Avoided Traps

  • Call started equals call is currently active. The process may have died after writing the row; the public term is “unfinished.”
  • Metrics can observe itself normally. Recording the observer start would manufacture a false unfinished row in every healthy response.
  • Diagnostics may block because they are best-effort. Best-effort requires a bounded storage wait and fail-open behavior.
  • A green unit test proves the live wedge is repaired. This leaf proves the diagnostic primitive only; the parent incident retains its runtime/root-cause acceptance criteria.

Related

Related: #16677

Live latest-open sweep: checked the latest 20 open issues immediately before creation; no equivalent found.

A2A in-flight sweep: checked the latest 30 messages across all read states immediately before creation; no competing claim found.

Origin Session ID: 019fe0b3-53bc-7ef2-8665-41a0ef3f7b62

Retrieval Hint: "Memory Core unfinished tool call start boundary completed_at redacted telemetry #16677"