LearnNewsExamplesServices
Frontmatter
id16707
titleSeat-cost reporter + wake-carried session context line
stateClosed
labels
enhancementaiperformanceagent-os
assigneesneo-kimi-iris
createdAtAug 8, 2026, 4:37 PM
updatedAtAug 8, 2026, 10:11 PM
githubUrlhttps://github.com/neomjs/neo/issues/16707
authorneo-kimi-phoebe
commentsCount4
parentIssue16682
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 8, 2026, 10:11 PM

Seat-cost reporter + wake-carried session context line

Closed Backlog/active-chunk-14 enhancementaiperformanceagent-os
neo-kimi-phoebe
neo-kimi-phoebe commented on Aug 8, 2026, 4:37 PM

Context

Split out of #16682 on 2026-08-08 so the wake context gate (PR #16705) can close as a clean one-PR leaf — draft PRs cannot receive formal reviews, and Resolves requires a fully delivered close target. The forensics, verdict, and evidence base live on #16682; this ticket is its instrumentation leaf.

The gate (#16682 PR) decides per wake; this leaf makes the cost regime visible in aggregate: per-seat per-day token spend from the two harness ledgers, the warm-window incidence that drives the needle, and the capped/uncapped ablation readout (Iris capped at 650K from 08-08; Phoebe the uncapped control).

The Problem

Today no seat and no operator can answer "what did a seat process today?" without hand-running ad-hoc scripts against two different harness stores (the 2026-08-08 drain forensics were exactly that: one-off Python against opencode.db and wire.jsonl). The numbers that matter — fresh input+output (the needle), cache-read, inter-call gap distribution vs the provider warm window — are all present in local telemetry and nowhere aggregated.

The Architectural Reality

  • Telemetry sources (both local files, no API dependency): opencode ~/.local/share/opencode/opencode.db message.data.tokens JSON (input / output / cache.read / cache.write); kimi-code ~/.kimi-code/sessions/*/agents/main/wire.jsonl usage records (inputOther / inputCacheRead / inputCacheCreation / output; consecutive duplicated usage lines must be deduped — wire lines double-record).
  • Home: ai/scripts/diagnostics/ beside the sibling diagnostics scripts; an ai: npm script entry. No new MCP tool (operator ruling 07-26: capability goes to buildScript/CLI/service methods).
  • Per-provider warm windows (Mnemosyne's input on #16682, reference-verified 08-08): K3 measured TTL cliff (degradation from ~20 min, ~0% after 1h); Anthropic default 5m / extended 1h (drops to 5m in overage); GPT unmeasured (column must render unmeasured, not invent a number).
  • The wake digest line (AC-2 below) consumes the probe result the gate already computes — no second probe.

The Fix

  1. ai: seat-cost reporter: per-seat per-day table (calls, fresh input, cache-read, output, est. needle%) aggregated from both ledgers; a gaps > warmWindow incidence column per seat (the warm-window map declared once in the reporter module — a graphless diagnostics script does not import AiConfig, mirroring the receiver's boundary); the ablation readout (capped vs uncapped seat deltas). Iris's burn-analysis/cache-ttl scripts are the basis — attribution in the module header.
  2. Wake-carried session cost (AC-4 re-homed): when the gate probed the session, the injected wake digest gains one line carrying the session's context size vs the gate threshold (e.g. [session-context: 45K tokens, gate at 250K]) — the seat sees its cost position on every wake, exactly where the #16540 wake discipline lives.

Acceptance Criteria

  • AC-1: The reporter reproduces the 2026-07-31→08-02 per-seat table (Phoebe 20.8M fresh / 715M cache / 1,714 calls; Iris 13.2M / 1,170M / 2,673 — see #16682) within rounding, from committed fixture data (fixture db + fixture wire.jsonl), including the wire dedupe.
  • AC-2: A wake delivered through a gated route whose probe succeeded carries the one-line session-context summary in the injected digest; absent probe data, no line (no noise).
  • AC-3: The gaps > warmWindow incidence column computes per-seat inter-call gap incidence against the per-family warm-window map; GPT renders unmeasured.
  • AC-4: Ablation readout: the capped (Iris, 650K) vs uncapped (Phoebe) per-day comparison is one flag, documented in --help.
  • AC-5: Unit specs for aggregation, dedupe, gap incidence, and the digest line (fixture-driven).

Out of Scope

  • The gate itself (PR #16705), thresholds, warm-window consumption in the delivery decision (#16682 discussion settled: no time-based flush).
  • Cross-checking against MC turn-presence (Mnemosyne's fleet-wide instrument) — follow-up once this stands.
  • Harness-side changes; dashboard reconciliation (#16682 AC-8, operator).

Related

  • #16682 (parent — forensics, verdict, gate)
  • #16539 / #16540 (wake-economy family)
  • Iris's data-pack comment on #16682 (IC_kwDODSospM8AAAABN37fPQ) — measurement method + scripts offered as the reporter core

Contract Ledger (repair-cycle authority, PR #16709)

Surface Contract
Source set: kimi-code ~/.kimi-code/sessions/<wd>/<session>/agents/main/wire.jsonl usage.record lines. The agents/main scope is contractual; subagent wires are excluded — census 2026-08-08: exactly 1 usage record across 32 non-main wires on the reference machine. Both usageScope values (turn, session) are summed; session-scope is 3 records fleet-history, turn-sized.
Source set: opencode ~/.local/share/opencode/opencode.db message rows, role === 'assistant' enforced at the parser boundary (parseOpencodeRows); the sqlite like pre-filter is a scan aid, not the contract. Token-bearing non-assistant rows are rejected (0 occur in today's live db; the boundary is spec-tested, not assumed).
Wire dedupe Consecutive records identical on every consumed field (incl. time and model) dedupe to one; a same-time/same-tokens pair whose model cell disagrees is a mid-session provider switch — two real records, both kept (spec-tested). Full-history census (untracked live corroboration): 0 consecutive dupes in the real ledgers; the arm is spec-exercised with synthetic pairs.
Provider family Every record preserves model identity (kimi model; opencode providerID/modelID). classifyProviderFamily: kimi/moonshot/k3 → kimi (20-min measured window); claude/anthropic → claude (5-min); gpt/openai → gpt (unmeasured); unknown → null (unmeasured). Seat family = modal record family (resolveSeatFamily). An unmeasured family renders unmeasured, never an inherited number — GPT-through-OpenCode included.
Output Markdown table: date, seat, calls, fresh input, cache read, cache write, output, raw total, est. needle*, gaps>warm. --json emits {seat: {family, harness, days}}.
Flags --from/--to YYYY-MM-DD — day window applied after bucketing, so a live boundary day includes the pre-window predecessor's gap; the committed fixture is bounded to its 4 days and asserts its own truth. --ablation — capped-vs-control needle lines for days ≥ cappedFrom. --json, --kimi-root, --opencode-db, --opencode-rows, --fixtures <dir>. --help/-h exits 0 with full usage; unknown args exit 1 with a usage hint.
Fixtures (privacy boundary) Deterministic synthetic records (test/playwright/unit/ai/scripts/diagnostics/fixtures/seatCost/syntheticFixtures.mjs; the spec materializes the fixture files into a tmp dir): 2,926 wire records + 1,920 opencode rows generated against the published #16682 table — per-call timestamps and token cells are generated (uniform splits, evenly-spaced synthetic times, deterministically placed over-window gaps), never real per-call telemetry. Reproduces the table exactly: iris 253/971/1,449 calls, phoebe 177/603/934; gap distribution iris 1/6/0, phoebe 3/9/7; both 08-08 = 2 (incl. the automatic 08-02→08-08 void gap). Live-ledger reproduction (digit-identical on the operator machine) + the zero-dupe census remain as untracked corroboration — real ledger projections are not committed (cycle-3 review boundary, PR #16709).
Wake digest line (AC-2) Gated route + successful probe + deliver → the dispatch-time envelope copy gains payload.sessionContext = {contextTokens, maxContextTokens}; formatLocalWakeDigest renders [session-context: <K> tokens, gate at <K>] directly under the [WAKE] header. Absent probe data → no field → no line (no noise). The durable record stays byte-identical to the signed envelope (copy semantics, spec-tested). Consumes the gate's existing probe result only — no second probe.

Origin Session ID: ses_01ed929e9ffe1kG4Ne612BUHhT (opencode, 2026-08-08)

Retrieval Hint: "seat cost reporter harness ledger aggregation warm window incidence"

tobiu referenced in commit 15a9212 - "feat(ai): per-seat session-cost reporter CLI (#16707) (#16709) on Aug 8, 2026, 10:11 PM
tobiu closed this issue on Aug 8, 2026, 10:11 PM