Retry transient GitHub REST failures in the DevIndex transport

Context

The scheduled Data Sync Pipeline on dev is currently failing before it can publish the refreshed DevIndex artifacts. Run 29541194846 aborted in Cleanup.reconcileRichUsersWithTracker() when getLoginByDatabaseId(5704244) reached GitHub REST and received 503 Service Unavailable.

The account lookup is valid: a fresh gh api user/5704244 resolves dregitsky. An earlier run, 29534893482, logged 267 REST 503 responses without retry before later failing on an unrelated generated-data rebase conflict. This is transient transport failure, not a deleted account.

Live latest-open sweep: checked the latest 20 open issues at 2026-07-17T00:11:17.148Z; no equivalent ticket found. The contemporaneous A2A claim sweep also found no competing DevIndex/GitHub REST retry lane.

The Problem

DevIndex.services.GitHub.rest() performs one fetch and throws every non-success response immediately. A single transient GitHub edge or proxy failure can therefore abort the complete Data Sync job, even when the same request would succeed moments later.

The caller must continue to fail closed after retry exhaustion: converting an unresolved identity lookup into null inside Cleanup would blur transient transport failure into account deletion and risk destructive reconciliation. The missing resilience belongs in the transport.

The DevIndex GitHub API guide currently describes retry behavior generically, although the implementation only retries GraphQL requests. It also uses a stale GraphQL database-ID example even though database-ID resolution is REST-owned.

The Architectural Reality

• apps/devindex/services/GitHub.mjs:258-286 owns the authenticated REST transport and is the narrowest correct retry boundary.
• apps/devindex/services/Cleanup.mjs:217 intentionally propagates unresolved identity failures; that identity-safety contract must remain unchanged.
• ai/services/github-workflow/GraphqlService.mjs already provides the proven local policy shape: bounded retries for 429/502/503/504 and recognized network failures, Retry-After support, capped exponential backoff, and jitter.
• test/playwright/unit/app/devindex/GitHubService.spec.mjs:57-63 proves exhausted transient failures propagate through getLoginByDatabaseId(), but does not exercise transport recovery.
• Retry policy values belong in documented, overrideable Neo static config fields. They must not become uppercase hidden magic numbers.

The Fix

Extend DevIndex.services.GitHub.rest() with a bounded, config-driven transient retry policy:

1. expose the attempt budget, base/max delay, jitter ratio, and retryable status set through documented static config fields;
2. retry 429, 502, 503, 504, and recognized fetch/network failures;
3. honor both numeric and HTTP-date Retry-After values, otherwise use capped exponential backoff with jitter;
4. retain immediate failure for non-transient responses and retain failure after the retry budget is exhausted;
5. add transport-level unit witnesses with zero-delay config overrides; and
6. correct the DevIndex GitHub API guide so its REST and GraphQL claims match production.

Contract Ledger Matrix

Decision Record impact

None. This is a local transport-resilience repair aligned with existing service precedent; it does not amend an ADR or change an external API contract.

Acceptance Criteria

• GitHub.rest() retries configured transient statuses including 429, 502, 503, and 504.
• Recognized fetch/network failures receive the same bounded treatment.
• Retry policy is documented and overrideable through Neo config fields; no hidden uppercase retry constants are introduced.
• Numeric and HTTP-date Retry-After values are honored; absent headers use capped exponential backoff with jitter.
• 404 and other non-transient 4xx responses remain fail-fast.
• Retry exhaustion still propagates, preserving Cleanup identity safety.
• Unit coverage proves 503-to-success, transport-error-to-success, exhausted budget, and no retry for a non-transient 4xx.
• learn/guides/devindex/data-factory/GitHubAPI.md accurately distinguishes GraphQL and REST retry behavior and replaces the stale database-ID example.
• Post-merge observation: confirm a subsequent scheduled Data Sync run traverses the repaired transport. External GitHub availability is not a merge gate.

Out of Scope

• Changing Cleanup fallback or identity semantics
• Adding workflow-level reruns
• Redesigning the GraphQL retry path
• Repairing generated-data rebase conflicts
• Changing quota-exhaustion policy
• Treating external GitHub availability as pre-merge proof

Avoided Traps

• Swallowing the 503 in Cleanup: would misclassify an unknown identity as a deleted account.
• Retrying the whole workflow: repeats expensive completed work and leaves every REST caller brittle.
• Hidden module constants: prevents operator/hardware-specific overrides and violates Neo config conventions.
• Unbounded retries: converts an upstream outage into a wedged pipeline.

Related

• #11585
• PR #11587
• Data Sync run 29541194846
• Data Sync run 29534893482

Origin Session ID: b681a37a-4353-4ed0-bbf1-b46e6f2501c7

Retrieval Hint: DevIndex GitHub REST 503 retry Data Sync transport Retrieval Hint: apps/devindex/services/GitHub.mjs rest Retry-After