Budget DevIndex updates by GraphQL cost

Context

A census of the latest 100 scheduled Data Sync Pipeline runs through 2026-07-23 07:37 UTC found 67 failures and 33 successes. Twelve of those failures terminated in DevIndex GitHub GraphQL quota or query-cost errors.

The workflow currently requests up to 800 user updates per run. That value originated as an explicitly temporary increase in f564430 and was never returned to the earlier safe batch size established by #9119.

Problem

DevIndex batch admission is expressed only as a user-count ceiling, while the scarce resource is GraphQL cost.

A single user can require a profile query, organization reads, and multiple contribution-year queries; long histories can split into additional bounded windows. Eight users are processed concurrently. Although the GitHub service records GraphQL rate-limit data, the updater's stop gate checks the REST core bucket instead. As a result, the pipeline can admit work when REST capacity is healthy but GraphQL capacity is already insufficient.

Retrying resource-limit responses can then multiply demand. An exhausted DevIndex phase can also leave no reserved capacity for the downstream content-index and SEO rebuild.

Architectural Reality

• The workflow sets --limit=800 before the downstream index rebuild: data-sync-pipeline.yml.
• The updater uses concurrency 8, accepts a numeric limit, and gates on rateLimit.core: Updater.mjs.
• Per-user work may fan out across multiple contribution periods: Updater.mjs.
• GraphQL responses already update the graphql bucket with cost, remaining, and resetAt: GitHub.mjs.
• The current configuration exposes user-count and concurrency-adjacent controls, but no GraphQL reservation policy: config.mjs.

Fix

Make GraphQL cost the admission-control currency for DevIndex updates.

• Keep the CLI --limit as a hard upper bound, not as a promise that all users will be admitted.
• Add a GraphQL budget/reservation policy to the existing GitHub service, based on the response-reported cost, remaining, and resetAt values.
• Reserve explicit capacity for downstream index/SEO work before admitting concurrent user updates.
• Let the updater consume the graphql bucket and stop/checkpoint gracefully before the reserve is crossed.
• Ensure concurrent admissions cannot oversubscribe a shared budget from the same snapshot.
• Treat a single-query resource-limit failure separately from primary rate-limit exhaustion; any window-splitting fallback stays bounded and budget-aware.
• Remove the temporary 800-user workflow default. Until cost-based admission is active, restore a conservative ceiling no higher than the historical 200-user batch.
• Emit compact telemetry that explains admitted users, observed cost, remaining capacity, reserve, and stop reason.

Contract Ledger

Decision Record Impact

none. This is an operational resource-governance correction within the existing DevIndex service boundary; it does not change stored data shape or repository ownership.

Acceptance Criteria

• A red test demonstrates that high REST capacity with low GraphQL capacity stops new DevIndex admission.
• GraphQL cost, remaining, and reset metadata drive a documented shared budget and downstream reserve.
• Concurrent workers cannot reserve more capacity than the shared budget permits.
• --limit remains a hard ceiling while actual admitted work is capacity-derived.
• The temporary 800-user workflow default is removed; rollout begins at a ceiling no higher than 200 until measured cost evidence supports increasing it.
• DevIndex stops/checkpoints cleanly before consuming the downstream reserve.
• Per-query resource-limit fallback is bounded, budget-aware, and distinct from primary quota exhaustion.
• Logs expose enough compact budget telemetry to diagnose the next scheduled run without replaying every query.
• Three scheduled runs that reach DevIndex complete without a GraphQL quota/query-cost terminal failure and preserve the declared downstream reserve.

Out of Scope

• GitHub App identity, cross-repository authorization, or ruleset bypass
• Regeneration after dev advances during a run
• Requesting a larger GitHub quota
• Changing DevIndex scoring or stored record shape
• Treating all GitHub 5xx responses as quota failures

Avoided Traps

• Lowering concurrency without a shared cost budget
• Checking the REST core bucket as a GraphQL proxy
• Treating user count as stable query cost
• Retrying resource exhaustion in a way that multiplies demand
• Reopening or mutating closed historical tickets

Related

• Authorization and publication identity sibling: #15744
• Historical safe batch: #9119
• GitHub API service groundwork: #9245
• Historical retry classification: #15359

Live duplicate sweep: the latest 30 open issues and recent all-state A2A claims were checked immediately before creation on 2026-07-23; no equivalent open lane was found.

Origin Session ID: 72bb1088-8ed5-48b7-a835-c288cf30e814

Retrieval Hint: Data Sync DevIndex GraphQL cost budget quota remaining reserve 800 concurrency