LearnNewsExamplesServices
Frontmatter
id12190
titlegh-workflow syncer: PR + Discussion delta-fetch + Discussion query cost + shared rate-limit governance
stateClosed
labels
bugaiperformance
assigneesneo-opus-ada
createdAtMay 30, 2026, 3:15 AM
updatedAtJun 6, 2026, 10:10 AM
githubUrlhttps://github.com/neomjs/neo/issues/12190
authorneo-opus-ada
commentsCount2
parentIssue12186
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 6, 2026, 10:10 AM

gh-workflow syncer: PR + Discussion delta-fetch + Discussion query cost + shared rate-limit governance

neo-opus-ada
neo-opus-ada commented on May 30, 2026, 3:15 AM

Context

Sub of #12186 (P1). Operator: GitHub rate limits hit more frequently. The wgr9t499x investigation attributed this to query shape, not item count — grounded here against the query files.

Findings (grounded)

  • Issues are correctly delta-gated: queries/issueQueries.mjs:42 ($since: DateTime) + :54 (filterBy: {since: $since}), driven by IssueSyncer.mjs:532. Smallest, cheapest fetch.
  • PR + Discussion queries have NO delta param — only issueQueries.mjs matches a since/filterBy grep. pullRequestQueries.mjs + discussionQueries.mjs full-scan the entire corpus every sync. The PR full-scan (~106 sequential pages per the investigation) is the dominant load on the secondary (requests/min) limit.
  • Discussion is the most expensive per-page shape: discussionQueries.mjs:102 nests replies(first: $maxReplies) under comments (~26 pts/page per the investigation).
  • Rate-limit governance is issue-path-only: rateLimit{…} + a low-remaining guard appear only in IssueSyncer.mjs (grep) — PR + Discussion runs are unguarded.

The Fix

  1. PR delta gating — fetch newest-first (orderBy: {field: UPDATED_AT, direction: DESC}) and stop paginating at the first item older than metadata.lastSync. (The pullRequests connection has no server-side since filter — confirm capability at implementation; pullRequestQueries.mjs:77 already references an intended $since.) Mirror the outcome of IssueSyncer.mjs:532.
  2. Discussion delta gating — same newest-first + cutoff pattern in DiscussionSyncer.
  3. Discussion query cost — paginate/gate the nested replies(first:$maxReplies) (continuation query, like the issue timeline) instead of inlining replies for the full comment set every run.
  4. Shared rate-limit governance — lift the rateLimit{cost,remaining,resetAt} accounting + low-remaining guard out of IssueSyncer into the shared GraphQL execution path so PR + Discussion are governed too.

Acceptance Criteria

  • PR + Discussion incremental syncs fetch only items updated since lastSync, not the whole corpus (full-scan → a handful of pages).
  • Discussion per-page point cost reduced (replies not inline-fetched for the full comment set every run).
  • The low-remaining rate-limit guard applies to all three syncers.
  • Issue delta gating unchanged (already correct).
  • A full reconcile / clean-slate still has an explicit path — the delta cutoff must not silently skip a genuine full re-sync.

Out of Scope

  • Bucketing correctness (#12188) + milestone guard (#12184).

Related

  • #12186 (epic). Authority: learn/agentos/decisions/0004-github-content-architecture.md.

Retrieval Hint: grep filterBy / $since / replies( / rateLimit in ai/services/github-workflow/{queries,sync}/.

tobiu referenced in commit aa9bc54 - "fix(github-workflow): PR + Discussion delta-fetch cutoffs (#12190) (#12193) on May 30, 2026, 8:05 AM