LearnNewsExamplesServices
Frontmatter
id14067
titlePullRequestSource per-element chunking (body + each review/comment) — #14033 PR vertical
stateClosed
labels
enhancementaiarchitecture
assigneesneo-opus-vega
createdAtJun 26, 2026, 7:42 AM
updatedAtJun 26, 2026, 9:38 AM
githubUrlhttps://github.com/neomjs/neo/issues/14067
authorneo-opus-vega
commentsCount0
parentIssue14033
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJun 26, 2026, 9:38 AM

PullRequestSource per-element chunking (body + each review/comment) — #14033 PR vertical

Closed v13.1.0/archive-v13-1-0-chunk-6 enhancementaiarchitecture
neo-opus-vega
neo-opus-vega commented on Jun 26, 2026, 7:42 AM

Context

#14033's PR vertical (sibling of the tickets vertical, PR #14065). PullRequestSource.extract() writes ONE whole-file chunk per pr-*.md (body + all reviews + comments combined). A multi-round PR (body + N reviews + M comments) trends toward the embedding cap → dropped / blind-byte-split (the #13999 metadata-without-vector class on the no-hard-skip path). Per-element chunks keep every element small.

The Fix

  1. Pure splitPullRequestArchiveMarkdown(content)[{kind:'body'|'review'|'comment', ordinal, content}]. Boundaries (V-B-A'd across 4121 PR files — 494 carry ## Reviews, 243 ## Comments): frontmatter + title + PR body (its OWN ## Deltas/## Test Evidence/## Commits sections stay in the body) until the FIRST of ## Reviews/## Comments; each discussion element is delimited by ### `@<author>` (<STATE>) reviewed on <ISO> (review) or ### `@<author>` commented on <ISO> (comment). No ## Reviews/## Comments → body-only.
  2. Wire PullRequestSource.extract() to emit pr-<id>#body + pr-<id>#review-<n> + pr-<id>#comment-<n> (stable names → idempotent re-ingestion), preserving type/kind/source per element + a per-element hash.

Acceptance Criteria

  • Splitter: body-only / reviews-only / comments-only / both-sections / element-with-internal-headings → correct elements.
  • extract() emits per-element chunks with stable names; type:'pull' + source preserved per element.
  • No-regression: a no-discussion PR → one body chunk equal to today's whole-file content.
  • Unit coverage for the splitter + the extract() per-element emission.

Out of Scope

  • DiscussionSource (thread format) — the third #14033 slice.

Contract Ledger Matrix

Consumed surface Field Change Consumer impact Fallback / semantics
PullRequestSource.extract() emitted KB chunks name per-element pr-<id>#body / #review-<n> / #comment-<n> (was pr-<id>) KB re-ingestion: one PR → N element chunks. name is metadata + the embedding-prefix, NOT a lookup/join key (verified for the identical consumers at #14063) → retrieval granularity improves, no name-keyed breakage. stable names → idempotent; old whole-file hash drops on re-ingest, per-element hashes added
same hash per-element content hash (was whole-file) re-ingestion replaces the one whole-file chunk with N element chunks content-fingerprint per element

Related

  • #14033 (parent — PR vertical; Discussion follows); #14063 / PR #14065 (tickets vertical — same pattern, different format); #14039 (v13.1 epic); #14000 (over-cap drop class).

Authored-by: @neo-opus-vega (Vega, Claude Opus 4.8). Session ef66cbd0-3770-466c-9df1-f93c141eb1d3.

tobiu closed this issue on Jun 26, 2026, 9:38 AM
tobiu referenced in commit e31c9a2 - "feat(ai): PullRequestSource per-element chunking — #14033 PR vertical (#14067) (#14069) on Jun 26, 2026, 9:38 AM