LearnNewsExamplesServices
Frontmatter
id13967
titleProve review-derived actionability before graph ingestion
stateClosed
labels
enhancementaitestingarchitecturemodel-experience
assigneesneo-gpt
createdAtJun 24, 2026, 10:22 PM
updatedAtJun 24, 2026, 11:49 PM
githubUrlhttps://github.com/neomjs/neo/issues/13967
authorneo-gpt
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJun 24, 2026, 11:49 PM

Prove review-derived actionability before graph ingestion

Closed v13.1.0/archive-v13-1-0-chunk-6 enhancementaitestingarchitecturemodel-experience
neo-gpt
neo-gpt commented on Jun 24, 2026, 10:22 PM

Context

We added explicit PR review templates so review feedback is structured and graph-ingestible. A concrete review body, #13915 review 4551887977, shows the value and the risk: it contains strong actionability signals such as Required Actions, an L0 blocker, related graph nodes, and later approval or neutralization. The current deterministic ingestor only consumes three bracket tags plus close-keyword issue links.

The key ROI question is not whether we can store review attributes. It is whether review-derived signals improve the Dream Pipeline ability to choose what to focus on next.

Release classification: post-release hardening. This is not a cloud deployment blocker.

The Problem

DreamPipeline.md and GoldenPathSynthesizer.mjs show that Golden Path routing uses frontier semantic proximity, ISSUE / DISCUSSION candidate vectors, inbound non-BLOCKS structural weight, and blocker or triage filters.

A generic PR_REVIEW node full of reviewer scores, approval states, reviewer names, and full review text would not directly enter the current routing candidate pool. Rubber-stamp approvals could also add misleading graph mass.

The useful surface is narrower: review-derived actionability that changes issue or discussion focus, blocker state, or durable gap evidence.

The Architectural Reality

Verified current substrate:

  • learn/agentos/DreamPipeline.md documents Golden Path scoring and the PR feedback ingestion contract.
  • ai/services/graph/GoldenPathSynthesizer.mjs scopes semantic candidates to ISSUE and DISCUSSION and scores inbound non-BLOCKS edge weight.
  • ai/services/ingestion/IssueIngestor.mjs#ingestPullRequestFeedback() upserts PULL_REQUEST, parses exactly [KB_GAP], [TOOLING_GAP], [RETROSPECTIVE], and links RESOLVES from close keywords.
  • ai/services/ingestion/ConceptDiscoveryService.mjs mines recent PR markdown into nodes.jsonl candidates, but those candidates begin validated:false and do not directly mutate Golden Path focus.
  • resources/content/pulls/chunk-5/pr-13915.md contains the concrete fixture: a first CHANGES_REQUESTED review with L0 Required Actions, then a later APPROVED review stating the L0 dependency landed and the prior blocker is resolved.

The Fix

Build this as ROI proof first, ingestion second:

  1. Create a focused fixture or proof around the synced #13915 review cycle.
  2. Define which review-derived signals are actionable enough to affect existing issue, discussion, or gap nodes.
  3. Reject raw review archiving as graph topology: approvals, reviewer names, full bodies, and score attributes must not increase priority by themselves.
  4. Only after the proof shows useful focus deltas, implement the smallest extraction path in the owning ingestion or synthesis substrate.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback / Edge Case Docs Evidence
Review-derived actionability proof DreamPipeline.md Golden Path scoring contract and GoldenPathSynthesizer.mjs candidate/score shape Demonstrate whether extracted review signals improve issue/discussion focus before adding parser topology If no useful focus delta is shown, no graph-ingestion expansion lands Ticket plus focused test/proof comments Unit/probe fixture using #13915 review cycle
PR review feedback ingestion Current IssueIngestor.ingestPullRequestFeedback() tag and RESOLVES behavior Extend only for actionable signals that map to existing issue/discussion/gap semantics Generic APPROVES, reviewer identity, full review text, and raw metric scores remain non-routing/audit-only JSDoc if behavior changes Focused ingestion tests, no Golden Path score pollution from approval-only reviews
Author response lifecycle tags pull-request review-response protocol If implemented, [ADDRESSED], [DEFERRED], [REJECTED_WITH_RATIONALE] map to Required Action lifecycle, not generic graph weight Missing or ambiguous mapping leaves the signal as KB/audit text only Protocol cross-reference only if parser lands Fixture covering addressed/deferred/rejected status extraction
Golden Path focus effect GoldenPathSynthesizer scoring model Review-derived blockers or gap signals affect existing issue/discussion/gap nodes in a bounded, explainable way Rubber-stamp approvals and score-only reviews produce zero structural boost DreamPipeline note only if behavior changes Before/after score assertion on a controlled fixture

Decision Record impact

None. This aligns with the existing Dream Pipeline and Native Edge Graph direction and does not amend an ADR.

Acceptance Criteria

  • A focused proof/test documents which #13915 review-cycle facts are actionable for Golden Path and which are noise.
  • The proof rejects generic PR_REVIEW attribute storage as routing weight unless it maps to a concrete issue/discussion/gap signal.
  • Approval-only or rubber-stamp review fixtures do not increase Golden Path structural weight.
  • REQUEST_CHANGES plus Required Actions can only affect focus when mapped to a concrete blocker/gap/follow-up target.
  • The later APPROVED / resolved review state can neutralize the earlier blocker signal in the same fixture, or the design explicitly leaves that signal non-routing.
  • If parser code lands, tests cover current [KB_GAP] / [TOOLING_GAP] / [RETROSPECTIVE] behavior so the existing contract does not regress.
  • If parser code does not land because ROI is unproven, the PR documents that outcome and leaves graph ingestion unchanged.

Out of Scope

  • Broad PR_REVIEW node ingestion.
  • Storing full review bodies in graph nodes.
  • Routing directly to pull requests from Golden Path.
  • Changing Golden Path ISSUE / DISCUSSION candidate scope before a proof shows it is useful.
  • Adding raw reviewer scores or approval edges as priority weight.

Avoided Traps

  • Rubber-stamp amplification: approval state is not trust. It must not become graph weight by itself.
  • Template theater: a structured review body is not automatically useful topology. The extracted signal must change focus quality.
  • Archival duplication: synced PR markdown and KB ingestion already preserve review text for retrieval; graph ingestion should not duplicate that as topology.

Related

Related: #13915

Duplicate / freshness evidence:

  • Live latest-open sweep: checked latest 20 open issues on 2026-06-24T20:19:40Z; no equivalent open ticket found.
  • A2A in-flight sweep: checked latest 30 all-state messages immediately before creation; no overlapping claim found.
  • Local exact/history sweep: found adjacent review-template validator and protocol work, but no ticket for Golden Path ROI proof before review-derived graph ingestion.
  • KB semantic sweep: ask_knowledge_base timed out; query_documents for review-derived actionability Golden Path PR review graph returned older lifecycle/protocol tickets, no equivalent open duplicate.

Origin Session ID: 30ba1931-9d59-48bb-afdf-af8904070af9

Handoff Retrieval Hints: query_raw_memories("review-derived actionability Golden Path PR review graph ROI #13915"); query_raw_memories("PR review template graph ingestion counterpart Required Actions metrics Evidence Audit").

tobiu closed this issue on Jun 24, 2026, 11:49 PM
tobiu referenced in commit e638362 - "test(ai): prove review actionability boundary (#13967) (#13973)" on Jun 24, 2026, 11:49 PM