LearnNewsExamplesServices
Frontmatter
id15755
titleAdd the disposable restore target-set meter
stateClosed
labels
enhancementaitestingperformance
assigneesneo-gpt-emmy
createdAtJul 23, 2026, 3:21 PM
updatedAtJul 23, 2026, 4:25 PM
githubUrlhttps://github.com/neomjs/neo/issues/15755
authorneo-gpt-emmy
commentsCount0
parentIssue15695
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJul 23, 2026, 4:25 PM

Add the disposable restore target-set meter

Closed Backlog/active-chunk-9 enhancementaitestingperformance
neo-gpt-emmy
neo-gpt-emmy commented on Jul 23, 2026, 3:21 PM

Context

#15695 is the long-lived measurement authority for restore-empty-target: it owns both the disposable instrument and the final 5k/20k receipt against the exact #15740 implementation head. Those two lifecycle boundaries cannot share one ready PR close target without closing the investigation before its authoritative receipt exists.

This leaf captures only the instrument implementation. Its PR may resolve this ticket while #15695 remains open for the exact-head measurement and merge-gate decision. No ticket needs to be reopened.

Live latest-open sweep: checked the latest 20 open issues at 2026-07-23T13:21:12Z; no equivalent implementation leaf exists. #15695 is the parent investigation and #15740 owns the actuator. The Knowledge Base surfaced the unrelated Gemma4 REM benchmark in #12074; it does not measure recovery target-set staging/promotion and is not a duplicate. The recent A2A claim sweep found only Emmy's existing #15695 meter claim.

The Problem

The final #15695 receipt cannot be produced until #15740 has a clean implementation head, but building the instrument only after that head exists would mix privileged recovery logic, resource instrumentation, fixture generation, and evidence-contract review into one critical-path PR.

The meter also needs to refuse evidence-class collapse. A deterministic file control can verify phase accounting, progress, batch maxima, resource aggregation, interruption, and reconciliation, but it cannot claim Chroma/SQLite scaling or satisfy the exact-head merge gate.

The Architectural Reality

ai/scripts/benchmark/ already owns executable measurement instruments; serving-cost-meter.mjs is the sibling CLI precedent. Pure report/aggregation logic belongs below ai/scripts/benchmark/helpers/, following servingCostCore.mjs, so it can be unit-pinned without starting services. Unit coverage belongs below test/playwright/unit/ai/scripts/benchmark/, following stats.spec.mjs.

The structure-map and structural pre-flight therefore resolve this as a matched sibling lift, not a new daemon, service, or recovery authority. ADR-0027 remains authoritative for the ordered memories → summaries → graph target-set lifecycle.

The Fix

Add:

  • ai/scripts/benchmark/restore-empty-target-meter.mjs: a run-owned CLI that creates disposable fixtures, samples resources, runs synthetic controls or an injected exact-head adapter, and emits a provenance-bearing JSON report;
  • ai/scripts/benchmark/helpers/targetSetMeasurementCore.mjs: a pure recorder that validates profile cardinalities, canonical phase order, progress, resource high-water marks, batch maxima, provider traces, and evidence class;
  • test/playwright/unit/ai/scripts/benchmark/TargetSetMeasurementCore.spec.mjs: focused contract and disposable-fixture coverage.

The fixed profiles contain 5,000 or 20,000 explicit-vector rows in each of memories and summaries. A deterministic 64-node / 63-edge graph remains constant across profiles so the instrument does not invent a one-to-one vector/graph relationship.

Contract Ledger

Target surface Source of authority Proposed behavior Fallback / refusal Docs Evidence
CLI --profile, --dimension, --control / --adapter #15695 investigation contract Create only internal OS-temp roots; run exactly one control or adapter mode Reject invalid profiles/options and refuse dirty exact-head worktrees Module JSDoc + --help Focused unit/integration spec + disposable 5k/20k controls
JSON report #15695 ACs + ADR-0027 phase order Preserve per-phase timings/counts, first/last progress, Node/process/disk high-water marks, batch maxima, provider trace, fixture receipt, and git identity Incomplete runs cannot report completed; every report keeps merge authority external Recorder JSDoc Pure recorder tests
Exact-head adapter seam #15740 owns persistent mutation Let the actuator drive the canonical recorder without importing action authority into the meter Clean 40-character head required; completed candidates require separable Chroma RSS evidence Adapter typedef/JSDoc Contract tests now; exact #15740 receipt later on #15695
Synthetic controls This ticket Exercise full, pre-promotion interruption, and post-first-promotion reconciliation paths using files only Always label synthetic-control, authoritative=false, mergeGateSatisfied=false CLI/module JSDoc 5k/20k control receipts; not production evidence

Acceptance Criteria

  • The CLI creates fresh run-owned bundle, staging, and production roots and never accepts a live Memory Core path
  • Fixed 5k/20k profiles and exact deterministic graph cardinality/serialized bytes are recorded
  • Admission, each staging phase, staged validation, ordered promotions, production revalidation, and terminal settlement have separate timings and progress receipts
  • Node heap/RSS, declared process RSS, logical temporary-file bytes, row counts, batch maxima, and provider-call trace are retained in the report
  • Full, interrupted pre-promotion, and post-first-promotion reconciliation controls emit truthful retained-resource checkpoints
  • Synthetic/seam evidence is mechanically non-authoritative
  • Exact-head adapter mode binds the report to a clean full git SHA and requires real separable Chroma sampling for a completed candidate
  • Focused unit coverage pins the report contract and disposable fixture primitives

Decision Record impact

aligned-with ADR 0027. This instrument measures the accepted target-set lifecycle and cannot select an action, mutate production, or redefine completion.

Out of Scope

  • Implementing #15740 or widening its action contract
  • Producing the final #15695 exact-head receipt
  • Opening or mutating live Memory Core, Chroma, or graph paths
  • Treating compact zero-vector file controls as production disk/RSS scaling evidence
  • Inventing wall-time or memory pass thresholds

Avoided Traps

  • Closing #15695 with the harness PR — would erase the unresolved exact-head gate and force a forbidden reopen later.
  • Mock-only success theatre — control reports remain non-authoritative even when green.
  • Node-only accounting — external and shared-process roles remain explicit, including a reason when RSS is not separable.
  • Action leakage — the adapter invokes an external owner; the benchmark does not implement recovery state transitions.

Related

  • Parent investigation: #15695
  • Exact action owner / final evidence dependency: #15740
  • ADR amendment: #15739
  • Source Discussion: #14032

Origin Session ID: 72bb1088-8ed5-48b7-a835-c288cf30e814

Retrieval Hint: "disposable restore target-set meter exact-head adapter synthetic control evidence class"