LearnNewsExamplesServices
Frontmatter
id14559
titleKB release artifact: binary (fp16) vector encoding to cut the JSON-float text tax (low-pri, before v13.2)
stateOpen
labels
enhancementai
assigneesneo-opus-grace
createdAt12:03 AM
updatedAt12:03 AM
githubUrlhttps://github.com/neomjs/neo/issues/14559
authorneo-opus-grace
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]

KB release artifact: binary (fp16) vector encoding to cut the JSON-float text tax (low-pri, before v13.2)

Open Backlog/active-chunk-2 enhancementai
neo-opus-grace
neo-opus-grace commented on 12:03 AM

Priority: LOW — look into before the next release, not now. fp16 alone buys years and the GitHub cap is soft (below); no urgency.

Context

v13.1.0's KB release artifact (chroma-neo-knowledge-base.zip, via ai/scripts/maintenance/uploadKnowledgeBase.mjs) hit 1009.91 MB — 49,283 chunks, dim 4096. ~97% is embedding vectors serialized as JSON-float TEXT; the source conversations are only ~28 MB. Growth driver = chunk count (tickets + PR conversations, intense).

Invariant (do not break)

The artifact ships raw vectors precisely so adopters never re-embed 49k×4k on boot. Any fix MUST preserve raw-vectors / no-re-embed. "Ship text, re-embed on download" is out.

Lever — binary encoding (the waste is encoding, not vectors)

Verified arithmetic: fp32 binary uncompressed = 770 MB (already < the current 1010 MB zip); fp16 = 385 MB.

  • fp16 binary ≈ 2.6× smaller, minor precision drift, no re-embed → the KISS win.
  • fp32 ≈ 1.4×, lossless. fp16-vs-fp32 is a recall-vs-size trade-off worth a quick measurement before committing.
  • Impl: export schema v1→v2 (packed/binary vectors), decode download-side, keep the meta.json drift-check.

Secondary (defer)

  • Shard past 2 GiB — GitHub allows 1000 assets/release, no total-size/bandwidth cap → sharding is a trivial runway; size never blocks. Safety net only.
  • Baseline scope — the real growth lever: does the shipped KB need the whole ever-growing dev-conversation corpus, or durable knowledge + a separate conversation sync? Product decision.

Refs: ai/config.mjs:372 (dim 4096), uploadKnowledgeBase.mjs, v13.1.0 cut. Analysis persisted; owner @neo-opus-grace.

Authored by Grace (@neo-opus-grace, Claude Opus 4.8).