Context
Euclid's exact-head intake for #15622 (issuecomment-5047376106) ran the named spec files on his live seat with the real NEO_* environment: SessionSummarization.spec.mjs's live model call measured 43,629 ms against the spec's hard 40,000 ms assertion — the only red in that file. This is a latency-threshold failure, not environment leakage; it had no ticket of its own.
The Problem
A hard wall-clock threshold on a model-latency assertion is non-deterministic by construction: model provider load, machine load, and cold/warm model state all move the number. A red from this assertion tells the reader nothing about whether summarization WORKS — only that this run was slow. If it stays a gate, it false-alarms on every loaded host (his seat, a busy CI runner, a laptop under load).
The Question
Is a deterministic latency gate on live-model summarization desirable at all? If yes, it needs a shape that measures something deterministic (batching behavior, call count, prompt shape, output contract) rather than elapsed time. If elapsed time is the intended signal, the threshold needs to be statistical (percentile over N runs) or environment-scaled, not a single hard number.
Acceptance Criteria
Out of Scope
- SourceRegistry audit ordering (owned by #15667 / PR #15715)
- Any environment scrubbing (#15622's narrowed framing)
Related
- #15622 (the intake that surfaced this as a distinct defect, issuecomment-5047376106)
test/playwright/unit/ai/...SessionSummarization.spec.mjs (the hard 40,000 ms assertion)
Live latest-open sweep: checked latest 20 open issues at 2026-07-22T14:28Z; no equivalent found. A2A in-flight sweep: no competing claim.
Origin Session ID: 72c8c42d-f18a-408c-97c8-aeb1f82dd276
Retrieval Hint: "SessionSummarization 40s latency threshold model call 43.6s non-deterministic gate"
Context
Euclid's exact-head intake for #15622 (issuecomment-5047376106) ran the named spec files on his live seat with the real
NEO_*environment:SessionSummarization.spec.mjs's live model call measured 43,629 ms against the spec's hard 40,000 ms assertion — the only red in that file. This is a latency-threshold failure, not environment leakage; it had no ticket of its own.The Problem
A hard wall-clock threshold on a model-latency assertion is non-deterministic by construction: model provider load, machine load, and cold/warm model state all move the number. A red from this assertion tells the reader nothing about whether summarization WORKS — only that this run was slow. If it stays a gate, it false-alarms on every loaded host (his seat, a busy CI runner, a laptop under load).
The Question
Is a deterministic latency gate on live-model summarization desirable at all? If yes, it needs a shape that measures something deterministic (batching behavior, call count, prompt shape, output contract) rather than elapsed time. If elapsed time is the intended signal, the threshold needs to be statistical (percentile over N runs) or environment-scaled, not a single hard number.
Acceptance Criteria
origin/dev) is recorded in the decisionOut of Scope
Related
test/playwright/unit/ai/...SessionSummarization.spec.mjs(the hard 40,000 ms assertion)Live latest-open sweep: checked latest 20 open issues at 2026-07-22T14:28Z; no equivalent found. A2A in-flight sweep: no competing claim.
Origin Session ID: 72c8c42d-f18a-408c-97c8-aeb1f82dd276
Retrieval Hint: "SessionSummarization 40s latency threshold model call 43.6s non-deterministic gate"