LearnNewsExamplesServices
Frontmatter
id10057
titleKnowledge Base: Add PullRequestSource for PR Conversation Embeddings
stateClosed
labels
enhancementai
assigneestobiu
createdAtApr 18, 2026, 12:48 AM
updatedAtApr 18, 2026, 8:44 PM
githubUrlhttps://github.com/neomjs/neo/issues/10057
authortobiu
commentsCount1
parentIssue9999
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtApr 18, 2026, 8:44 PM

Knowledge Base: Add PullRequestSource for PR Conversation Embeddings

tobiu
tobiu commented on Apr 18, 2026, 12:48 AM

Intent / The Why

The Knowledge Base indexes content from six source providers: ApiSource, LearningSource, ReleaseNotesSource, TicketSource, DiscussionSource, and TestSource. The GitHub Workflow server syncs PR conversations to local markdown files at resources/content/pulls/ (~280 files), but no PullRequestSource exists to embed them into the vector database.

This means:

  • ask_knowledge_base(query='how was the grid selection refactored?') can find the ticket that described the intent, but not the PR that documents the actual implementation, review feedback, and architectural corrections made during execution.
  • The PR body contains the richest signal: diff summary, reviewer scores, scope-creep warnings, and follow-up ticket links. Without embeddings, this knowledge is invisible to semantic search.

The Q&A Signal Pattern

The old workflow (pre-Swarm) put both problem and solution in ticket comments. With PRs, the signal split: ticket = problem, PR = solution. If the KB only indexes tickets, agents can find what to do but not how it was done before. This asymmetry degrades RAG quality over time as more work flows through PRs.

Scope of Work

New Source Provider

Create ai/mcp/server/knowledge-base/source/PullRequestSource.mjs following the DiscussionSource.mjs pattern:

  1. Scan resources/content/pulls/*.md
  2. Parse YAML frontmatter (number, title, author, state, mergedAt, head, base, url)
  3. Chunk the PR body + conversation into embeddable segments
  4. Tag each chunk with type: 'pull' metadata for filtered queries

KB Query Integration

The query_documents tool already supports type filtering. After adding PullRequestSource, queries with type='pull' should return PR content. No tool changes needed — just the source provider.

Key Files

File Action
ai/mcp/server/knowledge-base/source/PullRequestSource.mjs [NEW] Source provider
ai/mcp/server/knowledge-base/config.mjs [MODIFY] Register new source

Reference Pattern

ai/mcp/server/knowledge-base/source/DiscussionSource.mjs — this is the closest analog. It scans resources/content/discussions/, parses frontmatter, and chunks content. PullRequestSource should follow identical patterns with PR-specific metadata.

Acceptance Criteria

  • PullRequestSource extracts and embeds PR markdown files
  • query_documents(query='grid refactoring', type='pull') returns relevant PR chunks
  • ask_knowledge_base(query='how was issue #N resolved?') can find the corresponding PR body
  • Existing source providers unaffected (no regressions)

Origin Session ID: 144326a4-c1e8-4eee-80e4-b984f3c79ddc

tobiu added the enhancement label on Apr 18, 2026, 12:48 AM
tobiu added the ai label on Apr 18, 2026, 12:48 AM
tobiu assigned to @tobiu on Apr 18, 2026, 8:10 PM
tobiu
tobiu Apr 18, 2026, 8:11 PM

Input from Claude Opus 4.7 (Claude Code):

Graph wiring (ticket-intake)

  • Parent: #9999 — fits the "KB knows more about project work history" framing. (Originally #9999 §3 scoped PRs under the Macro DB plan; that vector-heavy direction was abandoned due to semantic dilution. This ticket delivers the same goal via the conventional chunked-source pattern instead.)
  • Related, not parent: #10030 — Concept Ontology is a different architectural layer (deterministic JSONL graph, not vectors). PR embeddings are complementary: concepts tell you what; PR embeddings surface how things got done.

Hypothesis vs root cause (flagged during intake)

The ticket prescribes "chunk the PR body + conversation into embeddable segments", but the actual DiscussionSource.mjs / TicketSource.mjs pattern is one chunk per markdown file — downstream chunking happens in DatabaseService. I will follow the established pattern (~40 LOC, mirror of DiscussionSource) rather than over-engineering per-section splitting.

Minor out-of-scope observations (not bundling into this PR)

  • TicketSource.mjs:14-17 JSDoc says .github/ISSUE_ARCHIVE but code reads resources/content/issues + issue-archive. Stale comment.
  • Frontmatter metadata is embedded in content rather than extracted across all source providers. Limits state='merged' style filters; fine for semantic search today.

Proceeding with branch agent/10057-pull-request-source from origin/dev.

tobiu added parent issue #9999 on Apr 18, 2026, 8:11 PM
tobiu referenced in commit 01f6a0f - "feat(kb): add PullRequestSource for PR conversation embeddings (#10057)" on Apr 18, 2026, 8:14 PM
tobiu cross-referenced by PR #10066 on Apr 18, 2026, 8:15 PM
tobiu referenced in commit 8e47e5f - "test(kb): move PullRequestSource spec to canonical unit/ path (#10057)" on Apr 18, 2026, 8:23 PM
tobiu referenced in commit 4e248dc - "test(kb): exercise PullRequestSource singleton against mock fs (#10057)" on Apr 18, 2026, 8:37 PM
tobiu closed this issue on Apr 18, 2026, 8:44 PM
tobiu referenced in commit fde93fe - "feat(kb): add PullRequestSource for PR conversation embeddings (#10057) (#10066) on Apr 18, 2026, 8:44 PM
tobiu cross-referenced by #9748 on Apr 20, 2026, 1:13 PM