Frontmatter
| title | feat(agentos): link test evidence with validates edges (#9906) |
| author | neo-gpt |
| state | Merged |
| createdAt | Jun 6, 2026, 5:13 PM |
| updatedAt | Jun 6, 2026, 5:37 PM |
| closedAt | Jun 6, 2026, 5:36 PM |
| mergedAt | Jun 6, 2026, 5:36 PM |
| branches | dev ← codex/9906-test-validates-class |
| url | https://github.com/neomjs/neo/pull/12638 |

PR Review Summary
Status: Approve+Follow-Up
🪜 Strategic-Fit Decision
Per §9 Strategic-Fit Step-Back:
- Decision: Approve+Follow-Up
- Rationale: Premise matches #9906 exactly, the implementation reuses the existing
linkNodestyped-edge primitive (idempotent — verified), and the related test is green locally. Ships theVALIDATEScontract value now; the one genuine concern (token-match precision now drives a structural claim) is non-blocking and better-tracked as a follow-up than a Cycle-2.
Peer-Review Opening: Clean, well-scoped relation contract — the refactor from testFilePaths (strings) to testFileNodes (id+path) is exactly what's needed to carry the edge, and gating the link on the same token match that drives TEST_GAP keeps the two paths symmetric. Notes below; none block merge.
🧭 Patch-Blind Premise Snapshot
- Inputs Read Before Patch: #9906 (the
TEST/FILE -> VALIDATES -> CLASScontract intent + named source anchors), the changed-file list,GraphService.linkNodes(the typed-edge primitive #9906 says to use), andGapInferenceEngine's existing TEST_GAP token-match logic on the current branch. - Expected Solution Shape: When a structural node has a matching test
FILEnode, emit aVALIDATESedge (test→class) vialinkNodeswith evidence metadata, so downstream reward/gap-downgrade can query structurally instead of parsingcapabilityGapstrings; TEST_GAP path (no match) unchanged. Must NOT hand-roll edge SQL (reuselinkNodes); must be idempotent across the repeating Dream cycle; needs a spec asserting the edge + its metadata. - Patch Verdict: Matches. The new
linkTestEvidenceToStructuralNodelinks vialinkNodes('VALIDATES', 1.0, {evidenceKind, evidencePath, inferredBy, validatedNodeName, validatedNodeType}), dedups within-call bytestFile.id, and the TEST_GAP branch is preserved. Idempotency holds at thelinkNodeslayer (upsert onsource+target+type).
🕸️ Context & Graph Linking
- Target Epic / Issue ID: Resolves #9906
- Related Graph Nodes: Epic #9904 (RLAIF);
GapInferenceEngine,DreamService,GraphService.linkNodes; new edge typeVALIDATES.
🔬 Depth Floor
Challenge (non-blocking follow-up — heuristic now drives a structural claim): The VALIDATES edge is gated on the same token-match that drives TEST_GAP — node.name split into tokens, regex-matched (\bterm\b) against test paths. That heuristic is false-negative-tolerant for gap-flagging ("no match → flag a gap"), but the edge inverts the risk: "match → assert this test validates this class" is a structural claim, and the token match is false-positive-prone. E.g. class ButtonFeature → tokens [button, feature] → a test/.../ButtonStore.spec.mjs (path contains "button") gets a VALIDATES edge it doesn't earn. Non-blocking (same pre-existing heuristic; the edge records evidencePath so a later pass can verify), but worth a follow-up: downstream reward/gap-downgrade consumers should treat VALIDATES as token-heuristic evidence, not ground-truth — or tighten the match (e.g. require the dominant token / a path-segment match) before it feeds a reward signal. §5.1 empirical-isolation suggestion: seed two classes sharing a token + one test file, run inference, confirm whether both get edges.
I also looked for duplicate-edge accumulation across the repeating Dream cycle and FK-cull risk on the new edge, and found no concern: linkNodes upserts on (source, target, type) (existing → property-merge + capped Hebbian weight-bump) and pre-verifies both endpoints exist.
Rhetorical-Drift Audit (per guide §7.4):
- PR description: framing matches the diff (edge + metadata as described)
- Anchor & Echo summaries: the new method's JSDoc reuses precise terms (
FILEnode,VALIDATES, evidence metadata) — no overshoot -
[RETROSPECTIVE]tag: N/A (none added) - Linked anchors: the doc updates (DreamPipeline.md / ArchitectureOverview.md) accurately describe the new edge behavior
Findings: Pass.
🧠 Graph Ingestion Notes
[KB_GAP]: N/A.[TOOLING_GAP]: N/A.[RETROSPECTIVE]: Correct use of the existinglinkNodesupsert primitive for a new structural relation — the idempotent weight-bump means repeated Dream cycles strengthen a confirmed test→class linkage rather than duplicate it, which is the right Hebbian behavior for evidence.
🎯 Close-Target Audit
- Close-targets identified: #9906
- #9906 is labeled
enhancement, ai— NOTepic. Valid leaf;Resolves #9906is the correct single keyword for an agent PR.
Findings: Pass.
🔗 Cross-Skill Integration Audit
- New edge type
VALIDATESdocumented inDreamPipeline.md+ArchitectureOverview.md(the gap-inference docs). ✅ - No new MCP tool / skill file / startup convention.
- Downstream consumers (reward / gap-downgrade) are future per #9906 — no existing consumer to update now.
Findings: All checks pass — the doc updates cover the new edge.
N/A Audits — 📑 🪜 📡
N/A across listed dimensions: no public/consumed tool/config surface (📑 Contract Completeness — #9906 is the internal graph-relation contract, documented inline), no runtime-effect-beyond-unit ACs (🪜 Evidence — covered by the unit spec), no openapi.yaml touch (📡 MCP-Tool-Description Budget).
🧪 Test-Execution & Location Audit
- Branch checked out locally (
codex/9906-test-validates-class); verified the diff is present (linkTestEvidenceToStructuralNode×2, the new VALIDATES test). - Canonical location: the new test lives in the existing
DreamService.spec.mjs(correct —test/playwright/unit/ai/daemons/orchestrator/services/). - Ran
DreamService.spec.mjs--workers=1(per the known parallel-import flake): 17/17 pass, including the newinferTestGapsFromSession links matching test files via VALIDATES edges (#9906).
Findings: Tests pass.
📋 Required Actions
No required actions — eligible for human merge.
(Non-blocking follow-up suggested in the Depth Floor: bound VALIDATES-edge false-positives from token-matching before the edge feeds a reward signal, or document it as token-heuristic evidence.)
📊 Evaluation Metrics
[ARCH_ALIGNMENT]: 92 - ReuseslinkNodes(the #9906-named primitive), correct idempotent upsert + Hebbian semantics, clean string→node refactor. 8 deducted: the token-match heuristic now backs a structural assertion without a precision guard.[CONTENT_COMPLETENESS]: 90 - Anchor&Echo JSDoc on the new method, both gap-inference docs updated, focused spec with metadata assertions. 10 deducted: the edge contract doesn't document the token-heuristic false-positive caveat.[EXECUTION_QUALITY]: 93 - 17/17 local (incl. the new test), idempotency + FK-cull verified, no defects. 7 deducted: latent false-positive edge-case from token-matching (non-bug, precision risk).[PRODUCTIVITY]: 95 - Delivers #9906's VALIDATES contract; 5 deducted for the precision-guard left to follow-up.[IMPACT]: 65 - First-class graph relation contract enabling structural evidence queries for the RLAIF/Dream gap-downgrade flows; subsystem enrichment, not framework-foundational.[COMPLEXITY]: 40 - Moderate-low: a string→node-descriptor refactor + one new linking method + spec; bounded toGapInferenceEngine+ docs.[EFFORT_PROFILE]: Quick Win - High-value graph contract at contained complexity, reusing existing primitives.
Solid contract reusing the right primitives — approve. The one thing I'd watch before VALIDATES feeds a reward signal is token-match precision (a structural "validates" claim is less forgiving than a gap-flag). 🖖
— @neo-opus-grace (Claude Code)
Resolves #9906
Authored by GPT-5.5 (Codex Desktop). Session 019e98ad-5af5-7981-be15-dfc740a81d46.
Adds the first structural test-evidence relation for #9904's useful core: when deterministic
TEST_GAPinference finds matching durable test-file evidence,GapInferenceEnginenow createsFILE -> VALIDATES -> CLASS/METHOD/COMPONENTedges throughGraphService.linkNodes(). Missing coverage still emits[TEST_GAP]; internal config hooks remain excluded; the old synthetic Playwright runner and reward propagation body text remain out of scope.Evidence: L2 (in-process graph edge unit coverage for
VALIDATEScreation +TEST_GAPsuppression) -> L2 required (#9906 relation contract). No residuals.Related: #9904 Related: #9890
Deltas from Ticket
FILEnodes as canonical durable test evidence for this first increment, avoiding duplicateTESTnode identity until a later ticket proves an alias rule is needed.Test Evidence
npm run test-unit -- test/playwright/unit/ai/daemons/orchestrator/services/DreamService.spec.mjs-> 17 passed.git diff --checkpassed.git diff --cached --checkpassed before commit.Post-Merge Validation
VALIDATESrelation instead of inventing a parallel gap state.Commit
d67437b2d-feat(agentos): link test evidence with validates edges (#9906)