LearnNewsExamplesServices
Frontmatter
title>-
authortobiu
stateMerged
createdAtApr 19, 2026, 4:59 PM
updatedAtApr 19, 2026, 5:12 PM
closedAtApr 19, 2026, 5:12 PM
mergedAtApr 19, 2026, 5:12 PM
branchesdevagent/10087-orphan-concept-channel
urlhttps://github.com/neomjs/neo/pull/10101
Merged
tobiu
tobiu commented on Apr 19, 2026, 4:59 PM

Summary

Promotes orphan concept detection from the ephemeral per-orphan logger.warn in ConceptIngestor to the durable capabilityGap tag channel + a dedicated sandman_handoff.md section — same pattern as [GUIDE_GAP] / [EXAMPLE_GAP] established in #10084. The next REM cycle and the next boot-loading agent session (Claude/Antigravity/Gemini) pick up the signal from the A2A context bridge; the Librarian daemon aggregates it during background maintenance. Incidental human auditors get it too, but they're the secondary consumer.

The Problem

#10084 (resolving #10035) shipped deterministic concept-graph gap detection for [GUIDE_GAP] and [EXAMPLE_GAP] via the capabilityGap channel. Orphan detection (concepts with no IMPLEMENTED_BY edge) was deferred to the ephemeral logger.warn channel in ConceptIngestor"orphans are ontology data-quality signals, not coverage gaps." That distinction is real but was the wrong place to draw it: DreamService runs offline, logger output scrolls off between cycles, and as the ontology grows (today's ~17 orphans will multiply with #10036 / #10037 / #10050 ingestion) the per-orphan log spam becomes noise while the signal itself stays invisible to the next boot-loading agent that reads sandman_handoff.md (per AGENTS_STARTUP.md §6) for strategic context.

This PR lands the architecture tobi and I aligned on during the #10085 ticket-intake review: share the capabilityGap channel with a new [ORPHAN_CONCEPT] tag, weight-gate the emission, render in a dedicated handoff section, and delete the logger.warn entirely.

Architectural Reality (verified against precedent)

  • ai/daemons/services/GapInferenceEngine.mjsinferConceptGraphGaps already iterates CONCEPT nodes and checks outbound EXPLAINED_BY / EXEMPLIFIED_BY edges. Adding a third branch for IMPLEMENTED_BY costs one filter call per concept — same cost as the existing two checks. The GUIDE_GAP_WEIGHT_THRESHOLD gate wraps all three signals so they share the same silencing discipline.
  • ai/daemons/services/ConceptIngestor.mjs:262 — orphan detection loop already exists for stats.orphansDetected. Deleting the logger.warn on the next line is a one-line change; the stat counter stays for the cycle-summary INFO log at line 272, which is the architecturally-correct place for phase-level observability.
  • ai/daemons/services/GoldenPathSynthesizer.mjs:254-316 — downstream consumer for the A2A plane. The parse + render pattern for [TEST_GAP] / [GUIDE_GAP] / [EXAMPLE_GAP] is a structural template: one tag branch in the parser + one if (arr.length > 0) block in the renderer. Extending for [ORPHAN_CONCEPT] mirrors this exactly — the 5-item limit and TTL pruning apply uniformly.
  • DreamService.spec.mjs:194+ — the existing GUIDE_GAP covered-concept fixture had EXPLAINED_BY + EXEMPLIFIED_BY but no IMPLEMENTED_BY. With the new signal, that "fully covered" test control would correctly emit [ORPHAN_CONCEPT], inverting the test's premise. Added IMPLEMENTED_BY to keep the control genuinely fully-wired.

Implementation Footprint

File Change
GapInferenceEngine.mjs inferConceptGraphGaps emits [ORPHAN_CONCEPT] when implementedByEdges.length === 0 && weight >= threshold. Class-level JSDoc updated to describe the three concept-graph signals + weight-gate rationale.
ConceptIngestor.mjs Deleted per-orphan logger.warn (~3 lines). Kept stats.orphansDetected++ for the cycle-summary INFO. JSDoc on syncConceptsToGraph updated to point at GapInferenceEngine as the surfacing home.
GoldenPathSynthesizer.mjs [ORPHAN_CONCEPT] parse branch + ### ⚠️ Orphaned Concepts render branch with same 5-item limit + TTL pattern.
DreamService.spec.mjs New [ORPHAN_CONCEPT] detection test (three cases: fully-wired, orphan-only, low-weight gated). synthesizeGoldenPath test extended with planted CONCEPT + ⚠️ section assertion. GUIDE_GAP covered-concept fixture gets IMPLEMENTED_BY edge.
ConceptIngestor.spec.mjs Orphan test now asserts stats.orphansDetected AND no per-orphan warn fires (AC3 enforcement).

Gold Standards Leveraged / Traps Avoided

  • Gold Standard (tag-prefix taxonomy): preserves the invariant from #10084 that all concept-level gaps land in the same capabilityGap property with distinct tag prefixes. One loop in GoldenPathSynthesizer, one TTL pruning path, uniform 5-item limit.
  • Gold Standard (weight gate on concept-level signals): reusing GUIDE_GAP_WEIGHT_THRESHOLD across all three signals keeps the tuning surface small and the semantics coherent. #10086 (config-lift) will move the single threshold, not three.
  • Gold Standard (else if branching for mutually-exclusive signals): GUIDE_GAP and EXAMPLE_GAP can't both fire (one requires EXPLAINED_BY absent, the other requires it present). Restructured the inner block to make the exclusion explicit while leaving ORPHAN_CONCEPT as an independent check. Behavior unchanged, readability up.
  • Trap avoided (logger as observability plane): per feedback_challenge_prescribed_fixes.md, logger output in an offline daemon is the wrong substrate for any signal a downstream agent needs to reason about across REM cycles. ORPHAN_CONCEPT now lives in the A2A plane (capabilityGap graph property → sandman_handoff.md section) where the next REM cycle's GoldenPathSynthesizer, the next boot-loading agent session, and the Librarian daemon all consume it durably.
  • Trap avoided (pre-emptive config-lift): kept the weight threshold as the existing named constant. If tuning becomes a real need, #10086 promotes it.

Testing

npm run test-unit -- test/playwright/unit/ai/services/ConceptService.spec.mjs test/playwright/unit/ai/daemons/

39 passed (ConceptService 23, DreamService 8 — one new ORPHAN detection test, DreamServiceGoldenPath 1, ConceptIngestor 7). Stable across 5 consecutive runs.

Behavioral test-of-success after merge:

  • sandman_handoff.md emits ⚠️ Orphaned Concepts for tier-1+ concepts without IMPLEMENTED_BY — visible to the next agent session booting the concept ontology context.
  • logger.warn output shrinks by one line per orphan per REM cycle (quieter daemon logs, durable signal).
  • Today's ~17 orphans + low-weight filter mean handoff stays mostly silent until #10036 / #10037 / #10050 enrich the ontology — the signal scales with the data.

Out of Scope / Follow-Up

  • #10086 — config-lift guideGapWeightThreshold to aiConfig.data. Now gates three signals instead of one; the value of config-lifting has grown slightly.
  • #10050 — concept description enrichment (blocked on human architect time).
  • #10036 / #10037 — Memory Core + ChromaDB concept ingestion that will populate more meaningful weights and make the orphan signal louder across the A2A plane.

Related

  • Parent: #10030 (Concept Ontology & Semantic Gap Inference)
  • Origin PR: #10084 (introduced the EXAMPLE_GAP pattern that this PR mirrors)
  • Preceding PR: #10100 (closed Items 1 + 2 of #10085; flagged Item 4 as belonging here)
  • Sibling: #10086 (config-lift threshold)

Resolves #10087

Origin Session ID: 1db25bbe-f39d-4dcd-bb0e-bc125ce91326

tobiu
tobiu commented on Apr 19, 2026, 5:05 PM

Input from Claude Opus 4.7 (Claude Code):

❋ # PR Review Summary

Status: Approved (pending human merge) — with a required correction to the PR body framing.

Self-Review Opening: Self-review of #10087. This PR chose (a) the shared capabilityGap channel + [ORPHAN_CONCEPT] tag over a new dedicated property (one loop, one TTL path, one weight gate — keeps the gap taxonomy coherent), (b) routing orphan surfacing through GapInferenceEngine rather than ConceptIngestor (all concept-level gap emission now lives in one place; ConceptIngestor is pure data ingestion), (c) an else if restructure of GUIDE_GAP/EXAMPLE_GAP to make mutual exclusion explicit while keeping ORPHAN_CONCEPT an independent check, (d) reuse of GUIDE_GAP_WEIGHT_THRESHOLD across all three signals rather than introducing a separate orphan threshold (one tuning surface, not three). Rationale documented in the PR body; deductions below focus on what didn't land cleanly — most importantly, a framing error tobi caught in the "Gold Standards / Traps Avoided" section.


📊 Evaluation Metrics

  • [ARCH_ALIGNMENT]: 90 — Reuses the proven tag-channel + render-section pattern from #10084's EXAMPLE_GAP. Weight-gate reuse keeps future tuning centralized (#10086 moves one threshold). else if restructure is a net readability win on the mutual exclusion. Deduction: the imprecise "humans" framing in the PR body (see Required Actions below) is evidence that I didn't fully internalize the A2A consumer identity of sandman_handoff.md while writing the rationale. The architectural decision is still correct — logger is ephemeral, graph+handoff is durable — but the framing suggests a fuzzy mental model of who actually reads the handoff.

  • [CONTENT_COMPLETENESS]: 76 — Anchor & Echo JSDoc is thorough (capabilityGap, sandman_handoff.md, REM cycle, GUIDE_GAP_WEIGHT_THRESHOLD, ORPHAN_CONCEPT, IMPLEMENTED_BY all echo across class + method JSDoc). PR body is structurally Fat-Ticket-compliant. Deductions, harder than I'd like:

    • (a) "Humans" framing wrong 3× in PR body. sandman_handoff.md is the A2A context bridge per AGENTS_STARTUP.md §6 (agents view_file on it at session boot) and the Fat Ticket protocol (Origin Session ID / REM handoff chain). Humans are incidental auditors, not the primary consumer. Lines currently saying "humans and future agents can act on them", "becomes invisible to humans reviewing sandman_handoff.md", "where humans see it" should instead frame the handoff's consumer correctly: the next REM cycle, the next boot-loading agent session (Claude/Antigravity/Gemini), the downstream Librarian daemon. This is a substantive framing error, not a tone issue — a human reviewer reading "where humans see it" might reasonably ask "but humans don't routinely read sandman_handoff.md — so why does this matter?", missing the A2A value.
    • (b) Missing test case: GUIDE_GAP + ORPHAN_CONCEPT co-emission (a concept with neither EXPLAINED_BY nor IMPLEMENTED_BY should emit both tags, not just one). My three-case test covers "fully-wired", "orphan-only (has guide+example)", and "low-weight gate", but doesn't verify the compound case. Low regression risk since the branches are independent if checks, but the compound assertion would make the orthogonality explicit.
    • (c) Post-merge KB sync isn't automated (#10088 tracks this recurring pattern).
  • [EXECUTION_QUALITY]: 85 — 39/39 stable across 5 runs. Clean diff (+187 / -48). else if restructure on the GUIDE_GAP/EXAMPLE_GAP block is semantically equivalent and a small readability win. Deductions:

    • (a) GUIDE_GAP covered-concept fixture broke on the first test run — the existing "fully covered" control concept had EXPLAINED_BY + EXEMPLIFIED_BY but no IMPLEMENTED_BY. With the new signal, that test immediately fired ORPHAN_CONCEPT on the control, inverting the test's premise. Evidence I didn't fully think through the new signal's interaction with the existing fixture landscape before implementing. Caught and fixed in the same session, but the gap should have been obvious from the check-what-breaks step.
    • (b) No symmetric afterEach in DreamService.spec.mjs per my freshly-saved feedback_symmetric_spec_cleanup.md. The spec's afterAll nullifies GraphService.db so cross-spec worker leak is bounded, but symmetric per-test cleanup is the discipline I just codified — I should apply it in the very next PR, not defer. Low-risk miss (no reproducible flake in 5 runs), but a missed opportunity to practice my own lesson.
  • [PRODUCTIVITY]: 92 — #10087's ACs satisfied, architecture matches the session discussion with tobi, scope stayed narrow. Logger.warn deletion is explicit in the diff. Test coverage maps to AC #5 ("ingesting an orphan concept causes ⚠️ Orphaned Concepts section to appear"). One compound-case test gap noted above — not blocking.

  • [IMPACT]: 55 — Signal promotion from ephemeral to durable. Closes the "Item 4 of #10085" loop. Today's thin ingestion + weight gate mean the handoff stays mostly silent until #10036 / #10037 / #10050 enrich the ontology. Architecturally load-bearing for the longer arc; not load-bearing on day 1.

  • [COMPLEXITY]: 35 — 3 library + 2 spec files, +187/-48, shallow scope. Reuses the #10084 pattern; no multi-service coordination beyond the existing Phase-0 ingest + cycle-scope gap inference chain.

  • [EFFORT_PROFILE]: Maintenance — follow-up polish on #10084's architectural pillar. Wrong-mechanism fix explicitly deferred from #10085 Item 4 and landed here per that ticket's scope reduction comment.


🕸️ Context & Graph Linking

  • Target Epic / Issue ID: Resolves #10087
  • Related Graph Nodes:
    • Parent: #10030 (Concept Ontology & Semantic Gap Inference)
    • Origin pattern: #10084 (EXAMPLE_GAP via capabilityGap)
    • Preceding: #10100 (Item 4 of #10085 deferred here by design)
    • Sibling: #10086 (config-lift — now more valuable; gates three signals not one)
    • Unblocks: cleaner orphan signal as #10036/#10037/#10050 enrich the ontology

🧠 Graph Ingestion Notes

  • [KB_GAP]: The A2A identity of sandman_handoff.md is implicit in AGENTS_STARTUP.md §6 and the Fat Ticket protocol, but not directly queryable via ask_knowledge_base("who consumes sandman_handoff.md?"). My framing error in the PR body is evidence that even after reading the init docs, the consumer identity wasn't sharp in my working model. Worth a future ticket: a dedicated learn/agentos/ doc explicitly stating "sandman_handoff.md is the A2A context bridge consumed by (1) the next REM cycle's GoldenPathSynthesizer, (2) the next boot-loading agent session, (3) the Librarian daemon during background maintenance; humans are incidental auditors, not the primary consumer."

  • [TOOLING_GAP]: None for this PR — the spec landscape I touched behaves well post-#10100's cross-spec leak fix. No new flakes observed.

  • [RETROSPECTIVE]: Second-order failure mode surfaced by tobi's challenge. My feedback_challenge_prescribed_fixes.md discipline held on the substrate dimension (correctly identified logger as ephemeral, graph+handoff as durable). But the consumer-identity dimension is an independent skill I didn't apply: "durable substrate" doesn't automatically tell you who reads it. The next layer of the discipline is: after picking the right substrate, correctly identify its consumer so the rationale and tone match reality. Filing feedback_observability_consumer_identity.md as a durable cross-session memory so this specific mistake doesn't recur. The skill-chain is now: challenge premise → challenge prescription → identify substrate → identify consumer. All four are discrete checks; passing one doesn't buy you the others.


📋 Required Actions

  • BLOCKING (editorial, not code): Edit PR body to correct the A2A framing in three places:
    • Summary: "Humans and future agents can act on...""The next REM cycle and the next boot-loading agent session pick up the signal in sandman_handoff.md, and the Librarian daemon aggregates it..."
    • The Problem: "becomes invisible to humans reviewing sandman_handoff.md""becomes invisible in the A2A context bridge agents boot from"
    • Gold Standards / Traps Avoided: "for any signal a human needs to act on. ORPHAN_CONCEPT now lives where humans see it.""for any signal a downstream agent needs to reason about. ORPHAN_CONCEPT now lives in the A2A plane — next REM cycle, next session boot, next Librarian pass."
  • (Optional, follow-up) Add GUIDE_GAP + ORPHAN_CONCEPT co-emission test case to make the branch orthogonality explicit.
  • (Optional, follow-up) Add symmetric afterEach to DreamService.spec.mjs per feedback_symmetric_spec_cleanup.md — apply my own fresh memory at the earliest opportunity rather than deferring.
  • (Post-merge, recurring) manage_knowledge_base sync so ask_knowledge_base("how does ORPHAN_CONCEPT surface?") returns current answers.

Approved for human merge once the PR body edit lands. 39/39 green, architecturally sound, honest deductions above. The "humans" framing was a real architectural-literacy miss — I'd rather own it explicitly here than pretend it's a wordsmithing nit.


tobiu
tobiu commented on Apr 19, 2026, 5:11 PM

Input from Claude Opus 4.7 (Claude Code):

❋ ## Polish commit pushed

Resolved both optional follow-ups from the self-review. 40/40 tests pass × 5 consecutive runs.

Changes

  • GUIDE_GAP + ORPHAN_CONCEPT co-emission test (DreamService.spec.mjs) — plants a tier-1 concept with zero outbound edges, asserts capabilityGap contains BOTH [GUIDE_GAP] AND [ORPHAN_CONCEPT] (and NOT [EXAMPLE_GAP], since that requires EXPLAINED_BY present). Locks in the branch orthogonality that the three-case test didn't hit directly: GUIDE_GAP/EXAMPLE_GAP are mutually exclusive, but ORPHAN_CONCEPT is independent.

  • Symmetric afterEach in DreamService.spec.mjs — mirrors the existing beforeEach graph-state cleanup. Applies the feedback_symmetric_spec_cleanup.md discipline from #10100: under fullyParallel: true Playwright interleaves specs in the same worker, and GraphService.db is a shared singleton. The existing afterAll nullifies GraphService.db so cross-spec leak was bounded at worker-shutdown, but per-test symmetric cleanup closes the door at the test-boundary level — aligned with the discipline I just codified and the earliest opportunity to apply it.

AC Status (final)

  • ConceptIngestor → no-op on orphan warn; stats retained
  • GapInferenceEngine emits [ORPHAN_CONCEPT] via capabilityGap channel (weight-gated)
  • GoldenPathSynthesizer renders ⚠️ Orphaned Concepts section (5-item limit + TTL)
  • Co-emission orthogonality locked in via compound test
  • Symmetric spec cleanup applied (first consumer of the feedback_symmetric_spec_cleanup.md discipline)
  • A2A framing corrected in PR body (no more "humans see it" — the next REM cycle / boot-loading agent / Librarian daemon is the consumer)
  • 40/40 tests pass, stable × 5 runs

Handoff back to you. PR ready for merge whenever. #10086 is the natural next pickup per our plan.