LearnNewsExamplesServices
Frontmatter
id12969
titleDecouple the KB artifact download from npm prepare — keep the zip opt-in
stateClosed
labels
enhancementaibuild
assigneesneo-fable
createdAtJun 12, 2026, 8:42 AM
updatedAtJun 12, 2026, 1:18 PM
githubUrlhttps://github.com/neomjs/neo/issues/12969
authorneo-fable
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 12, 2026, 1:18 PM

Decouple the KB artifact download from npm prepare — keep the zip opt-in

Closed v13.1.0/archive-v13-1-0-chunk-1 enhancementaibuild
neo-fable
neo-fable commented on Jun 12, 2026, 8:42 AM

Context

Operator recommendation during the v13 cut, after the #12967 forensics: "we can keep the zip, for users that want the qwen3-8b vectors, without reparsing. BUT, npm preinstall should no longer include it."

Release classification: post-release (operator-scoped; the published v13 artifact and install flow work — this removes a newly-expensive default).

The Problem

package.json:89 chains the KB artifact fetch into the npm prepare lifecycle: "prepare": "husky && node ./ai/scripts/setup/initServerConfigs.mjs && node ./ai/scripts/maintenance/downloadKnowledgeBase.mjs". prepare fires on every npm install in the repo — every fresh contributor/agent checkout (and CI environments that don't suppress lifecycle scripts) now downloads + ingests the v13 artifact, which is 583MB (vs 2.27MB at 12.1.0; the #12967 forensics decomposed why: ×26 corpus growth — 46.4% tickets/PR-conversations/discussions — × 4,096-dim qwen3-8b vectors × JSON-text float encoding).

The artifact itself stays valuable AND stays as-is per the operator: consumers who want the pre-computed qwen3-8b vectors without reparsing/re-embedding get exactly that. The problem is solely the default-on coupling to install time.

The Architectural Reality

  • package.json:89 — the prepare chain (husky + server configs + KB download). The first two members are cheap and install-appropriate; the third is now a half-gigabyte network+ingest operation.
  • package.json:35the opt-in command already exists: "ai:download-kb": "node ./ai/scripts/maintenance/downloadKnowledgeBase.mjs". The fix is removal + pointers, not new machinery.
  • ai/scripts/maintenance/downloadKnowledgeBase.mjs — fetches the collection-scoped release artifact and ingests it (no-reembed import per DatabaseService.importDatabase).

The Fix

  1. Remove downloadKnowledgeBase.mjs from the prepare chain (keep husky + initServerConfigs).
  2. Sweep onboarding/setup docs and any agent boot flows that implicitly rely on prepare-time KB ingestion; point them at npm run ai:download-kb as the explicit step (candidates: learn/agentos/** setup guides, .agents/ onboarding payloads, the cloud-deployment example's workspace setup).
  3. A friendly absence hint: when KB-dependent surfaces (e.g. ask_knowledge_base) find an empty collection, the error/log should name the one-liner (npm run ai:download-kb) — turning the removed default into a discoverable opt-in.

Acceptance Criteria

  • npm install on a fresh checkout performs no KB download/ingest.
  • npm run ai:download-kb stays wired as the opt-in (package.json:35 untouched).
  • Doc/boot-flow sweep done; empty-collection surfaces hint the opt-in command.

AC re-scope (cycle 2, PR #12979): the v13-asset round-trip verification moved to #12967's verification matrix — its artifact rework ships v1-auto-detect import, whose verification inherently covers the v1 asset; a separate pre-rework round-trip of the soon-replaced path is wasted motion (and a live import into a shared unified-Chroma dev instance is exactly the ⭐ B4 live-DB-bleed hazard ADR 0019 warns about — it belongs in #12967's isolated verification setup). Split per the Resolves-mandate shape (operator rule #12367: partial-delivery tickets get split, not Refs'd).

Contract Ledger

Consumed surfaces this change touches (delivery = PR #12979 @ 997136b49):

# Contract Before After Consumers Compat risk Verification
1 npm prepare lifecycle (package.json:89) husky + initServerConfigs + downloadKnowledgeBase (583MB fetch+ingest) husky + initServerConfigs only fresh checkouts; CI without --ignore-scripts; agent workspaces; container builds Deliberate behavior removal: install no longer populates the KB — cold-start = empty collection (mitigated by #3) live npm run prepare demo on the branch: only husky + initServerConfigs ran
2 npm run ai:download-kb (package.json:35) the opt-in command byte-identical — untouched consumers wanting pre-computed qwen3-8b vectors without reparse none package.json diff; asset round-trip rides #12967's matrix (v1-auto-detect, isolated Chroma)
3 ask_knowledge_baseSearchService#ask answer, empty-collection branch generic "No relevant documents found" names the one-liners (ai:download-kb / ai:sync-kb) via a collection.count() === 0 probe; degrades to the generic message on any probe error every agent/harness consuming ask; portal surfaces answer shape unchanged ({answer, references[]}) — only the string content of one branch differs SearchService.spec 13/13: empty-hint, probe-error→generic, populated-no-match all pinned
4 SearchService query/list paths raw structured shapes untouched — hint is deliberately ask()-only (structured consumers stay prose-free) programmatic consumers none spec populated-no-match branch; PR Deltas section records the scoping rationale
5 DeploymentCookbook.md fresh-clone flow implied install-time KB population documents the deliberate non-population + the explicit opt-in step operators / deployers docs-only review
6 Residual split AC2 round-trip gated here #12967's verification matrix release verification n/a #12967 AC-addendum comment (live)

Install-time egress contract: 583MB → 0 per npm install (the change's purpose); the artifact itself stays as-built (#12967 owns its future shape).

Out of Scope

  • Artifact size/format/scope work — #12967 owns that (the two tickets compose: this one removes the default cost, that one shrinks the opt-in cost — and now also owns the asset round-trip verification).

Related

#12967 (the artifact forensics + size levers + round-trip verification), #12696 (the cut where this surfaced).

Live latest-open sweep: checked latest open issues at 2026-06-12T06:55Z; no equivalent found. A2A in-flight sweep: clean.

Origin Session ID: e605ce21-3668-445c-bc00-45896aa9a092

Retrieval Hint: "npm prepare downloadKnowledgeBase opt-in decouple 583MB"

tobiu referenced in commit 8c22108 - "feat(ai): KB artifact download leaves npm prepare — explicit opt-in (#12969) (#12979) on Jun 12, 2026, 1:18 PM
tobiu closed this issue on Jun 12, 2026, 1:18 PM