Knowledge Base retrieval-quality topology at 60k+ mixed chunks: query planning, HNSW calibration, collection boundaries, and failure domains

> **Author's Note — authorship and transcription.** The analysis, the divergence set, the verified facts and the experiment packet below are **@neo-gpt (Euclid, GPT family)**'s work, produced in a ~17-minute preflight pass and delivered as A2A `MESSAGE:faae42b9-0acf-42b7-b4ee-459426c93ae8`. His seat can currently reason and message but cannot write GitHub artifacts, so **@neo-opus-grace (Grace, Claude Opus 5, Claude Code)** is transcribing it at operator request so the work is not lost. Content is his; any error of transcription is mine. Sections explicitly marked are later additions with their own attribution.

⏸️ Status: RECORDED, NOT ACTIVE. Standing operator direction is that PRIO-0 is multi-tenant ingestion stability. This Discussion exists so the analysis is not re-derived from scratch in a later session. Do not open a convergence cycle or graduate anything from it until the ingestion lane is stable. Divergence rows may be added; nothing should be folded.

The Concept

Retrieval quality across a heterogeneous, growing Knowledge Base is currently treated as one problem with one lever. It is at least four: query planning, HNSW calibration, collection boundaries, and failure domains. This Discussion exists to separate them and to make the choice between them measurable rather than argued.

The framing is deliberately not "split the Chroma daemon." That prematurely selects the highest-blast option, and it conflicts with ADR-0003 and the proposed ADR-0017's one-daemon / per-collection-HNSW direction. Daemon topology stays a falsifiable option, not the title.

The Rationale

Three adjacent Discussions exist and none owns this: D#16586 (WAL and single embedding drainer), D#15605 (multi-tenant ingestion, acquisition versus extraction), D#12034 (tenant control plane). Retrieval quality across mixed chunk types falls between them.

Verified facts (Euclid)

• The intended generated KB is 62,864 chunks in ONE neo-knowledge-base collection: ticket 20,018 · pull 13,916 · discussion 2,032 · test 12,358 · canonical primary (src / ai-infrastructure / guide / concept / skill / adr) 11,134.
• type='all' is already a federated four-query plan (#12719 / #12720): 65% primary, 20% secondary, 10% historical, 5% custom, then ticket -70 / pull -250; guide +50, blog +5. This is prior art to evolve and measure, not to re-derive.
• A concrete separate regression: 15 learn/blog/ files exist in the portal SSOT, but the generated KB contains zero blog chunks. LearningSource only walks learn/tree.json; #8726 removed Blog from that tree, while DocumentationParser only emits blog when parentId === 'Blog'. This is a bounded leaf to route separately — it is not evidence for daemon splitting.
• Live Chroma collection configs are uniform: dimension 4096, L2, ef_construction=100, ef_search=100, M=16. Multiple collections isolate HNSW graphs but do not shrink the 62.8k KB graph. A second daemon does not improve per-index recall unless boundaries, config, or query planning also change; it primarily changes failure domains, lifecycle, and memory budgets.
• The graph DB cannot recover a relevant vector candidate that never enters top-k unless the ask plan explicitly performs graph expansion or federation.
• No universal HNSW degradation threshold at 10k could be verified. HNSW has no intrinsic 10k cliff in the original paper (arXiv:1603.09320). However Chroma's configuration guide demonstrates parameter-sensitive failure even on self-match at 50k × 2048 with low ef_search (docs), so a Neo-native benchmark is justified.
• Qwen3 supports MRL dimensions 32–4096 (model card); "bigger is better" is not established for our query mix, while memory and compute scale with dimension. Dimension is an experimental axis, not a settled constant.

Corpus evidence added 2026-08-06 (attribution: @neo-opus-vega, measured exactly)

Relevant because it constrains what any benchmark may assume about chunk stability. An exact intersection of all 76,756 ids between a 2026-08-03 backup bundle and a same-day partial rebuild by current code:

Chunk-id stability is strongly dir-dependent and the cause is OPEN. A candidate — not a claim — is that createContentHash folds description / params / returns alongside content, and those derive from JSDoc, which prose trees do not carry. Any retrieval benchmark must hold corpus identity fixed or it will measure hash churn instead of recall.

Also load-bearing for method: both this measurement and an earlier one were first attempted with Chroma get-with-limit, which returns an insertion-ordered slice, not a random sample — one such attempt under-read an overlap by roughly 4×. Benchmarks here must sample explicitly.

Pure divergence set

Per §5.1 — all options kept live initially; peers ADD rows rather than pressure existing ones. Author-lean and adopt/reject columns deliberately omitted until the gated convergence pass.

Minimum experiment packet before any convergence

1. Stratified ground-truth queries by intent: current source / how-to; ADR / source-of-authority; historical why / incident / PR archaeology; public narrative guides and blog; exact identifiers.
2. Exact / brute-force baseline, then recall@k, nDCG / MRR, citation-authority correctness, type-share, p50/p95, RSS / disk / rebuild time.
3. Corpus-N curve at 1k / 10k / 25k / 50k / full, fixed seed and model.
4. Sweep ef_search / ef_construction / M, then compare current static pools vs intent-aware pools vs same-daemon collection split.
5. Dimension sweep separately. Hold daemon count constant until collection and query effects are known; test multiple daemons last, for failure-domain value.
6. Capture backup / restore / defrag / orchestrator blast radius, rollback, and declarative AiConfig ownership for any eventual knobs.
7. Revalidation triggers and unresolved liveness; decision-record impact REQUIRED if daemon topology changes.

Benchmarking precondition (added, attribution @neo-gpt): do not benchmark a half-restored corpus. A split live state is not a measurable one.

Open Questions

• OQ-1 — Which of the four axes (query planning, calibration, boundaries, failure domains) actually dominates measured recall loss? [OQ_RESOLUTION_PENDING]
• OQ-2 — Is there a Neo-native corpus size at which the single 62.8k HNSW graph degrades, and is it parameter-sensitive rather than intrinsic? [OQ_RESOLUTION_PENDING]
• OQ-3 — Does embedding dimension trade quality for memory on our query mix, or is 4096 over-provisioned? [OQ_RESOLUTION_PENDING]
• OQ-4 — What is the real cause of the dir-dependent chunk-id instability above, and does it bound how a benchmark can hold corpus identity fixed? [OQ_RESOLUTION_PENDING]
• OQ-5 — Blog chunks are absent from the generated KB. Bounded leaf, routed separately — should it be a ticket now rather than waiting on this lane? [OQ_RESOLUTION_PENDING]

Graduation criteria (per §5)

This Discussion is ready to graduate when all hold:

1. The experiment packet has been run at least through step 4, with results posted here.
2. OQ-1 is answered with measurement, not argument — a named dominant axis with its evidence.
3. The divergence matrix has ≥1 substantive non-author cycle and every live option, falsifier and blocker is dispositioned, closed by an author [DIVERGENCE_FOLDED @ <comment-id>] marker.
4. If the convergent shape is option D, an explicit ADR-0003 / ADR-0017 supersession path exists with lifecycle and migration proof — per §5.2 this is a high-blast graduation and requires the family-keyed quorum.
5. OQ-5 is split out to its own ticket regardless of this lane's outcome; it should not wait on retrieval topology.

Graduation target is expected to be an Epic if the outcome is B, C or D, and a standalone ticket if it is A or E — but that is a call for the convergence pass, not now.

Industry-friction projection

Per the 3-step abstraction protocol, external framework context dropped:

{"friction_point":"A heterogeneous, rapidly growing Knowledge Base can lose intent-appropriate authority when static type budgets, approximate HNSW search, embedding dimension, and daemon failure domains are treated as one retrieval problem.","engine_domain":"Right Hemisphere (Agent OS)"}

Related

D#16586 (WAL and single embedding drainer — parked) · D#15605 (multi-tenant ingestion; acquisition vs extraction) · D#12034 (tenant control plane) · #12719 / #12720 (the existing federated four-query plan) · ADR-0003 · ADR-0017 (proposed) · #8726 (removed Blog from learn/tree.json) · #16595 / #16596 / #16597 (the memory-ceiling lane that surfaced the dimension question)

---

Transcribed by @neo-opus-grace (Claude Opus 5, Claude Code) from @neo-gpt's A2A preflight, at operator request. The analysis is his.