Budget DevIndex updates by GraphQL cost

Context

A census of the latest 100 scheduled Data Sync Pipeline runs through 2026-07-23 07:37 UTC found 67 failures and 33 successes. Twelve of those failures terminated in DevIndex GitHub GraphQL quota or query-cost errors.

The workflow currently requests up to 800 user updates per run. That value originated as an explicitly temporary increase in NEO_INLINE_CODE_0 and was never returned to the earlier safe batch size established by #9119.

Problem

DevIndex batch admission is expressed only as a user-count ceiling, while the scarce resource is GraphQL cost.

A single user can require a profile query, organization reads, and multiple contribution-year queries; long histories can split into additional bounded windows. Eight users are processed concurrently. Although the GitHub service records GraphQL rate-limit data, the updater's stop gate checks the REST core bucket instead. As a result, the pipeline can admit work when REST capacity is healthy but GraphQL capacity is already insufficient.

Retrying resource-limit responses can then multiply demand. An exhausted DevIndex phase can also leave no reserved capacity for the downstream content-index and SEO rebuild.

Architectural Reality

• The workflow sets --limit=800 before the downstream index rebuild: NEO_INLINE_CODE_3.
• The updater uses concurrency 8, accepts a numeric limit, and gates on rateLimit.core: NEO_INLINE_CODE_6.
• Per-user work may fan out across multiple contribution periods: NEO_INLINE_CODE_7.
• GraphQL responses already update the graphql bucket with cost, remaining, and resetAt: NEO_INLINE_CODE_12.
• The current configuration exposes user-count and concurrency-adjacent controls, but no GraphQL reservation policy: NEO_INLINE_CODE_13.

Fix

Make GraphQL cost the admission-control currency for DevIndex updates.

• Keep the CLI --limit as a hard upper bound, not as a promise that all users will be admitted.
• Add a GraphQL budget/reservation policy to the existing GitHub service, based on the response-reported cost, remaining, and resetAt values.
• Reserve explicit capacity for downstream index/SEO work before admitting concurrent user updates.
• Let the updater consume the graphql bucket and stop/checkpoint gracefully before the reserve is crossed.
• Ensure concurrent admissions cannot oversubscribe a shared budget from the same snapshot.
• Treat a single-query resource-limit failure separately from primary rate-limit exhaustion; any window-splitting fallback stays bounded and budget-aware.
• Remove the temporary 800-user workflow default. Until cost-based admission is active, restore a conservative ceiling no higher than the historical 200-user batch.
• Emit compact telemetry that explains admitted users, observed cost, remaining capacity, reserve, and stop reason.

Contract Ledger

Decision Record Impact

none. This is an operational resource-governance correction within the existing DevIndex service boundary; it does not change stored data shape or repository ownership.

Acceptance Criteria

• A red test demonstrates that high REST capacity with low GraphQL capacity stops new DevIndex admission.
• GraphQL cost, remaining, and reset metadata drive a documented shared budget and downstream reserve.
• Concurrent workers cannot reserve more capacity than the shared budget permits.
• --limit remains a hard ceiling while actual admitted work is capacity-derived.
• The temporary 800-user workflow default is removed; rollout begins at a ceiling no higher than 200 until measured cost evidence supports increasing it.
• DevIndex stops/checkpoints cleanly before consuming the downstream reserve.
• Per-query resource-limit fallback is bounded, budget-aware, and distinct from primary quota exhaustion.
• Logs expose enough compact budget telemetry to diagnose the next scheduled run without replaying every query.
• Three scheduled runs that reach DevIndex complete without a GraphQL quota/query-cost terminal failure and preserve the declared downstream reserve.

Out of Scope

• GitHub App identity, cross-repository authorization, or ruleset bypass
• Regeneration after dev advances during a run
• Requesting a larger GitHub quota
• Changing DevIndex scoring or stored record shape
• Treating all GitHub 5xx responses as quota failures

Avoided Traps

• Lowering concurrency without a shared cost budget
• Checking the REST core bucket as a GraphQL proxy
• Treating user count as stable query cost
• Retrying resource exhaustion in a way that multiplies demand
• Reopening or mutating closed historical tickets

Related

• Authorization and publication identity sibling: #15744
• Historical safe batch: #9119
• GitHub API service groundwork: #9245
• Historical retry classification: #15359

Live duplicate sweep: the latest 30 open issues and recent all-state A2A claims were checked immediately before creation on 2026-07-23; no equivalent open lane was found.

Origin Session ID: 72bb1088-8ed5-48b7-a835-c288cf30e814

Retrieval Hint: Data Sync DevIndex GraphQL cost budget quota remaining reserve 800 concurrency