LearnNewsExamplesServices
Frontmatter
id15020
titleServing-cost meter: preserve requested-window semantics on early stop
stateClosed
labels
bugaitesting
assigneesneo-gpt
createdAt2:58 AM
updatedAt3:48 AM
githubUrlhttps://github.com/neomjs/neo/issues/15020
authorneo-gpt
commentsCount0
parentIssue15014
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAt3:48 AM

Serving-cost meter: preserve requested-window semantics on early stop

Closed Backlog/active-chunk-5 bugaitesting
neo-gpt
neo-gpt commented on 2:58 AM

Context

Found during exact-head re-review of PR #15013 / ticket #15014 after its four formal review actions were resolved. The meter deliberately supports Ctrl-C as a graceful early stop and still writes a provenance report, but the corrected boundary-coverage contract exposes one contained residual in that path.

The Problem

ai/scripts/benchmark/serving-cost-meter.mjs sets windowBounds.endMs = Date.now() after the sampling loop, while aggregateWindow() documents those bounds as the requested window and the metric identity remains rolling-window-${options.window}.

Direct falsifier: an invocation requesting 1h, interrupted after roughly 22.5 seconds, wrote windowMs: 3600000 but a nominal windowBounds duration of only roughly 22.5 seconds. With two or more samples, the report can therefore emit rolling-window-1h metric bags whose coverage ratio is calculated against the short observed interval. The artifact is provenance-rich but semantically mislabels an interrupted run.

This does not affect naturally completed windows and the report is not auto-ingested, so it is safe follow-up debt rather than a merge blocker for #15013.

The Architectural Reality

  • The pure aggregation helper already has the correct ownership boundary: callers inject the requested windowBounds; it reads no clock.
  • The CLI entrypoint owns wall-clock lifecycle and must keep requested versus observed time distinct.
  • An early stop is missing coverage, not a shorter requested window.
  • The tenant ingestion path remains separate; this ticket does not add automatic graph writes.

The Fix

Keep the requested end as startedAt + windowMs for aggregation and metric coverage. Record the actual stop separately in the report (for example observedEndMs plus interrupted: true). Naturally completed runs retain their existing shape/semantics. Preserve the low-coverage disclaimer so an interrupted run cannot masquerade as a complete measurement.

Acceptance Criteria

  • A Ctrl-C-interrupted 1h run keeps a nominal requested window of 3,600,000ms and records the actual stop separately.
  • Leading/trailing unavailable duration and coverage ratio are computed against the requested window; a short interrupted run cannot present as a fully covered rolling-window-1h.
  • Naturally completed runs remain unchanged apart from additive explicit lifecycle metadata, if any.
  • Focused coverage exercises the early-stop boundary and the normal-completion boundary without a wall-clock-duration test.
  • Cross-family review.

Out of Scope

The real 24-hour / second-hardware measurement runs and billing-console read owned by parent #14687 · automatic business-metric ingestion · pricing claims.

Decision Record Impact

None — localized measurement-contract correction aligned with the landed pure-helper boundary.

Duplicate / Freshness Evidence

Live open-issue search for “serving cost Ctrl-C requested window interrupted” returned no match. Latest-open queue checked immediately before creation; #15014 is the delivered instrument ticket, not this interrupted-window residual. Memory prior-art sweep surfaced the #15013 build lineage but no existing follow-up.

Origin Session ID: 019f484c-662f-7f31-969a-cbde373efd4a Retrieval Hint: "serving cost meter Ctrl-C requested window observed end rolling-window coverage"

tobiu referenced in commit f1950ea - "fix(ai): preserve requested serving-cost window on interrupt (#15020) (#15025)" on 3:48 AM
tobiu closed this issue on 3:48 AM