LearnNewsExamplesServices
Frontmatter
id15149
titleMeasure community-activity shadow coverage and cost
stateClosed
labels
enhancementaitestingarchitectureperformance
assigneesneo-gpt
createdAtJul 14, 2026, 7:29 AM
updatedAtJul 19, 2026, 11:09 AM
githubUrlhttps://github.com/neomjs/neo/issues/15149
authorneo-gpt
commentsCount0
parentIssue15145
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[x] 15148 Define community-activity authority in ADR 0036
blocking[x] 15150 Add tenant-scoped community source registration
closedAtJul 18, 2026, 9:11 PM
milestonev13.2

Measure community-activity shadow coverage and cost

Closed Backlog/active-chunk-5 enhancementaitestingarchitectureperformance
neo-gpt
neo-gpt commented on Jul 14, 2026, 7:29 AM

Context

Discussion #15139 and Grace's STEP_BACK require shadow instrumentation as the first implementation leaf after ADR 0036. The 30-day sample is explicitly a lower bound; it cannot justify production cadence, retention, pagination, TTL, steward, or wake values.

This is one fully closeable PR leaf under Epic #15145. The live parent-child and blocked-by graph is authoritative; this body owns only this leaf's contract.

The Problem

The current source mix hides API cost, missing resource families, duplicate/revision rates, storage growth, and attention amplification. Coding thresholds now would turn a measured unknown into permanent policy.

The Architectural Reality

The one-shot measurement path matches ai/scripts/maintenance/probeBusinessMetrics.mjs plus its pure core sibling. Provider reads remain owned by ai/services/github-workflow/; results are evidence only and must not admit events, create Tasks, advance production checkpoints, or wake residents.

The Agent OS structure map was run on 2026-07-14. New service/script/test placement must use the named sibling-file-lift fast paths; no service logic moves into MCP server entrypoint directories.

The Fix

Add a reproducible shadow probe and report schema that exhausts named GitHub resource families without mutating community authority. Record provider pages, candidate observations, actor kinds/trust cohorts, duplicates, revisions/tombstones, coverage gaps, latency, projected bytes, and the hooks needed for later admission/attention measurements.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback / Edge Case Docs Evidence
Shadow probe CLI Discussion #15139 OQ10 Read-only reproducible run with explicit window/source coordinates Partial provider access yields degraded coverage, never a completeness claim CLI help + measurement guide Unit fixtures plus recorded report
Measurement report OQ10 metric list JSON-first evidence with denominators and lower-bound flags Unknown metrics remain null/unknown Schema JSDoc Fixture replay and two-run variance evidence
Authority firewall Grace STEP_BACK point 5 No Task, wake, admission, checkpoint, or policy mutation Failure leaves production state unchanged Inline guard Negative side-effect tests

Decision Record impact

Depends on accepted ADR 0036; aligned with ADR 0035's evidence-not-authority firewall.

Decision Record

Required: ADR 0036. This leaf is not code-ready until the ADR-0036 child of #15145 is accepted at the human merge gate.

Discussion Criteria Mapping

Upstream graduated criterion This leaf's executable contract
OQ2 Measures the resource-by-event-family coverage matrix and honest gaps.
OQ10 Measures volume, cost, latency, duplication, revision, storage, and amplification inputs.
STEP_BACK point 5 Is the first implementation leaf and leaves every operational threshold unset.

Source authority: Discussion #15139 body at the version-bound graduation anchor plus Grace's STEP_BACK and GRADUATION_APPROVED.

Operator Scope Clarification — 2026-07-14

The community substrate is not a mirror of every GitHub repository notification. It separates:

  1. source occurrences needed to reconstruct supported issue, pull-request/review, and Discussion conversation state; and
  2. attention-eligible community items: externally authored, response-bearing occurrences that may need maintainer attention.

Stars/un-stars, forks, watches, and equivalent popularity telemetry are outside the community-event source families and cannot enter Bird View, counts, wake, or Task claim. Internal/rostered actions may update or resolve the state of an existing external item without minting new community attention. First-time versus trusted-repeat external status affects trust/projection, not basic eligibility. Bot eligibility must be an explicit ADR disposition and cannot be inferred from provider actor kind or trust tier.

Attention eligibility remains zero-authority: it does not assign work, enter LifecycleFrontier, or create a Task. Only the explicit canonical claim transition owns that promotion.

Acceptance Criteria

  • AC1 — A reproducible shadow command emits a versioned JSON-first report and human summary.

  • AC2 — Issue, PR/review, and Discussion families expose per-family pages, rows, gaps, and mutation evidence.

  • AC3 — Actor kind and security/content-trust cohorts remain separate measurements.

  • AC4 — Duplicate/update/tombstone rates, latency, and projected storage are measured or explicitly unknown.

  • AC5 — The report labels lower bounds and degraded/incomplete coverage; it never claims unseen history.

  • AC6 — No production event, checkpoint, Task, count projection, or wake is mutated.

  • AC7 — At least two repeated runs document variance and source-manifest identity.

  • AC8 — No cadence, retention, pagination cap, TTL, steward, or wake threshold is introduced.

  • AC9 — The report separates all supported source occurrences from attention-eligible external items and reports the exclusion reason by actor/source family.

  • AC10 — Negative fixtures prove star/un-star, fork, watch, and equivalent popularity telemetry do not inflate candidate rows, projected storage, counts, or wake estimates.

Out of Scope

Production source registration, durable admission, connector scheduling, policy selection, or wake delivery.

Avoided Traps

Do not turn the probe into the completeness authority, persist prose, infer deletions from absence, or smuggle defaults into config.

Related

Origin Session ID: 837ad74b-c2d2-413d-9aab-b7165a93a82a

Handoff Retrieval Hints

  • community activity shadow probe API pages duplicate revisions
  • Discussion 15139 OQ10 measurement lower bound

Creation Freshness

Creation duplicate sweep: immediately before filing at 2026-07-14T05:29:58.166Z, checked the latest 20 open issues and last 30 all-state A2A messages. The independent broader audit at 2026-07-14T05:13:00Z covered open and closed issues, pull requests, A2A, ADRs, and code; no equivalent owner or foreign claim existed.