Frontmatter
| id | 9088 |
| title | Fix: DevRank Spider Premature Exit (Split Core/Search Rate Limits) |
| state | Closed |
| labels | bugai |
| assignees | tobiu |
| createdAt | Feb 10, 2026, 5:48 PM |
| updatedAt | Feb 10, 2026, 6:05 PM |
| githubUrl | https://github.com/neomjs/neo/issues/9088 |
| author | tobiu |
| commentsCount | 2 |
| parentIssue | 8930 |
| subIssues | [] |
| subIssuesCompleted | 0 |
| subIssuesTotal | 0 |
| blockedBy | [] |
| blocking | [] |
| closedAt | Feb 10, 2026, 6:00 PM |
Fix: DevRank Spider Premature Exit (Split Core/Search Rate Limits)
tobiu added parent issue #8930 on Feb 10, 2026, 5:51 PM

tobiu
Feb 10, 2026, 5:57 PM
Input from Gemini 3 Pro:
✦ I have implemented the rate limit splitting and verified it works.
Changes:
- Rate Limit Buckets:
GitHub.mjsnow trackscore,search,graphql, andintegration_manifestlimits separately.- Smart Parsing: The
x-ratelimit-resourceheader is parsed to update the correct bucket.- Targeted Checks:
Spider.mjs:runSearchcheckssearchbucket.processRepositoriescheckscorebucket.Updater.mjs: Explicitly checkscorebucket (as GraphQL generally consumes this quota or has its own high limit).- Logging: Logs now clearly indicate which bucket is critical (e.g.
RATE LIMIT CRITICAL (core)).Result: The Spider successfully processed ~260 candidates before hitting the Core rate limit (403 Forbidden on
raspberrypi/linux), correctly triggering the shutdown for the Core bucket. It did not exit prematurely due to the Search limit. The fix is effective.

tobiu
Feb 10, 2026, 5:57 PM
Input from Gemini 3 Pro:
✦ I will close this ticket now as the fix is verified.
tobiu closed this issue on Feb 10, 2026, 6:00 PM
tobiu assigned to @tobiu on Feb 10, 2026, 6:05 PM
Problem
The
DevRank.services.Spiderincorrectly triggers a "Graceful Exit" (stopping all operations) when the GitHub API Search Rate Limit (30 req/min) is reached, even if the Core Rate Limit (5000 req/hr) is still healthy.This happens because
GitHub.mjscurrently tracks a singlerateLimitobject, which gets overwritten by the headers of the last request.rateLimitupdated to ~29/30.rateLimit.remainingis 29 (from Search) and incorrectly thinks it's below the safety threshold (50), triggering a shutdown.Goal
Decouple rate limit tracking to respect GitHub's
x-ratelimit-resourceheader.Proposed Changes
Refactor
GitHub.mjs:rateLimitconfig to store multiple buckets:rateLimit: { core: { remaining: 5000, reset: null, limit: 5000 }, search: { remaining: 30, reset: null, limit: 30 }, graphql: { remaining: 5000, reset: null, limit: 5000 }, integration_manifest: { remaining: 5000, reset: null, limit: 5000 } }#updateRateLimitto readx-ratelimit-resourceand update the corresponding bucket.Update
Spider.mjs:GitHub.rateLimit.searchforrunSearch.GitHub.rateLimit.coreforfetchContributors/processRepositories.Update
Updater.mjs:graphqlorcore).Acceptance Criteria