LearnNewsExamplesServices
Frontmatter
id9042
titleEnhancement: Dynamic High Star Ranges for Spider
stateClosed
labels
enhancementai
assigneestobiu
createdAtFeb 7, 2026, 10:20 PM
updatedAtFeb 7, 2026, 10:21 PM
githubUrlhttps://github.com/neomjs/neo/issues/9042
authortobiu
commentsCount1
parentIssue8930
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtFeb 7, 2026, 10:21 PM

Enhancement: Dynamic High Star Ranges for Spider

Closed v12.0.0 enhancementai
tobiu
tobiu commented on Feb 7, 2026, 10:20 PM

Enhance the "Core: High Stars" strategy in the Spider to avoid repeatedly scanning the same top-level repositories.

Current Behavior: The strategy uses a fixed query stars:>1000 (or configured minStars). This always returns the same top repositories (React, Vue, etc.), which are likely already visited, resulting in wasted runs.

Goal: Implement "Deep Slicing" by using random star count ranges. This allows the Spider to "jump" into the middle of the dataset (e.g., repositories with 1200-1500 stars) and treat them as "Page 1" of a new search, effectively bypassing the 1000-result limit of GitHub search API and discovering repositories that would otherwise be on Page 50+.

Logic: Instead of stars:>1000:

  1. Pick a random lower bound (e.g., 1000 + random(0..10000)).
  2. Pick a random upper bound (e.g., lower + 1000).
  3. Query stars:LOWER..UPPER.
tobiu added the enhancement label on Feb 7, 2026, 10:20 PM
tobiu added the ai label on Feb 7, 2026, 10:20 PM
tobiu added parent issue #8930 on Feb 7, 2026, 10:20 PM
tobiu assigned to @tobiu on Feb 7, 2026, 10:20 PM
tobiu referenced in commit 0e3bbb6 - "enhancement: Dynamic High Star Ranges for Spider (#9042)" on Feb 7, 2026, 10:21 PM
tobiu
tobiu Feb 7, 2026, 10:21 PM

Input from Gemini 3 Pro:

✦ Enhanced the "Core: High Stars" strategy to use dynamic star ranges (Slicing).

Changes

  • Replaced the static query stars:>1000 with a dynamic range generator.
  • Logic:
    • Lower Bound: 1000 + random(0..19000)
    • Upper Bound: Lower + 1000 + random(0..2000)
    • Query: stars:LOWER..UPPER

Impact

This allows the spider to randomly "jump" into the middle of the repository list (e.g., repositories with 1500-2000 stars) and scan them as Page 1. This effectively bypasses the GitHub search API limit of 1000 results and prevents the spider from repeatedly scanning only the top 30 most popular repositories.

Code committed to dev branch.

tobiu closed this issue on Feb 7, 2026, 10:21 PM