LearnNewsExamplesServices
Frontmatter
id17070
titleA truncated embedding is stored as a valid vector: the provider reports truncation, Neo never reads it, and no floor links per-slot context to the embedding safe band
stateOpen
labels
bugairegressionperformanceagent-os
assignees[]
createdAt10:25 PM
updatedAt10:25 PM
githubUrlhttps://github.com/neomjs/neo/issues/17070
authorneo-opus-vega
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]

A truncated embedding is stored as a valid vector: the provider reports truncation, Neo never reads it, and no floor links per-slot context to the embedding safe band

Open Backlog/active-chunk-15 bugairegressionperformanceagent-os
neo-opus-vega
neo-opus-vega commented on 10:25 PM

Context

ai/configBase.mjs:829 declares the embedding input band:

safeProcessingLimitTokens: leaf(28672, 'NEO_LOCAL_MODELS_EMBEDDING_SAFE_PROCESSING_LIMIT_TOKENS', 'number')

Neo will send a single embedding input of up to 28,672 tokens. For Qwen3-Embedding-8B that is deliberate — the model emits a 4096-dimension vector intended to represent a whole file, so the input has to be the whole file. A vector computed from a prefix does not represent the document it is indexed under.

llama-server reports per request whether it had to cut the input:

slot release: id 2 | task 2 | stop processing: n_tokens = 9144, truncated = 0

The Problem

Two independent gaps that compose into silent corruption.

1. Nothing links the engine's per-slot context to Neo's own safe band. contextTokensPerSlotRequired is supplied per lane profile and is validated only against itself (createProviderLaneSupportedLimitRequest throws SUPPORTED_LIMIT_BATCH_INVALID when requestedContextTokens < requiredTokens). That check is sound but circular: it enforces whatever the profile declared, never that the declaration is ≥ safeProcessingLimitTokens. A profile declaring a per-slot context below 28,672 passes every existing gate while guaranteeing that inputs Neo itself classifies as safe will not fit.

2. Neo never reads truncated. A repo-wide search of ai/services/ and ai/daemons/ finds truncation handling only in graph-walk helpers (laneLandscapeCensusSource, conceptNeighborhoodProbe). Nothing in the embedding path consumes the flag the provider returns.

The combination is the defect. When an input exceeds per-slot context, the provider does not fail — it truncates, returns HTTP 200, and returns a well-formed 4096-dimension vector. Neo stores it. The document is now indexed by a vector describing only its first N tokens.

This is worse than the failure modes we have been chasing. A failed embed is loud: it increments counters, trips backoff, degrades health, and eventually gets investigated. A truncated embed is silent, permanent, and indistinguishable from a correct one at every later stage — retrieval returns plausible-looking results, ranked by a vector that never saw most of the file. There is no counter that goes up and no surface that turns red.

Corpus damage is also not self-healing: re-ingesting the same content under the same shape reproduces the same truncated vector, and nothing marks the affected rows as suspect.

Architectural Reality

  • The provider is the only party that knows whether truncation happened; the caller cannot infer it from the response shape, which is valid either way.
  • LLAMA_ARG_ENDPOINT_SLOTS is already enabled on the shipped lane, so per-slot context is queryable at runtime — the floor can be checked before any content is embedded, not merely inferred after.
  • safeProcessingLimitTokens already exists as the authoritative statement of how large an input Neo will send. The provider-lane profile simply does not consult it.
  • KB_TENANT_REPO_SYNC_CONTENT_NOT_EMBEDDABLE already exists for chunks refused before the provider. There is no counterpart for content the provider silently accepted and cut.

The Fix (shape)

  1. Derive or validate contextTokensPerSlotRequired against safeProcessingLimitTokens. A lane whose per-slot context is below the safe band must fail composition, not run.
  2. Read the truncation signal and treat a truncated embedding as a failure, never as a stored vector. Whatever the transport reports (truncated, n_tokens vs input length) must reach the caller.
  3. Verify the floor at boot against the live engine, so a deployment whose per-slot context is too small is rejected before it writes a single vector rather than after it writes thousands.
  4. Give truncation its own reason code (sibling to CONTENT_NOT_EMBEDDABLE) so the condition is countable and distinguishable from a provider outage.
  5. Consider a corpus audit path: given a lowered floor, which stored vectors were computed under it? Without this, a fixed floor still leaves silently-wrong rows in place.

Acceptance Criteria

  • A lane profile whose contextTokensPerSlotRequired is below safeProcessingLimitTokens fails composition with a reason naming both numbers.
  • Boot compares live per-slot context against the safe band and refuses to embed when the engine cannot hold a safe-band input.
  • An embedding response indicating truncation is surfaced as a typed failure; no truncated vector is ever persisted.
  • Truncation has a distinct reason code and is counted separately from provider timeouts and refusals.
  • Negative control: an input inside the band on a correctly-sized lane embeds normally with no new failure path.
  • Regression: a fixture engine with per-slot context below the band, given a safe-band input, produces a typed truncation failure rather than a stored vector.

Out of Scope

  • Chunking strategy — this ticket is about honouring the band Neo already declares, not changing it.
  • Admission ordering (#17062), ledger leaks (#17064), actuator deadlines (#17065).

Avoided Traps

  • "The election already validates context." It validates the profile against itself. Nothing ties the profile to safeProcessingLimitTokens, so a too-small declaration is consistent and wrong.
  • "HTTP 200 means it worked." For embeddings it means only that a vector was returned. Correctness of a vector is not observable from its shape — which is exactly why the provider's own truncation report is the only signal available.
  • "Truncated vectors are still better than nothing." They are worse than nothing: they are indistinguishable from correct ones, so they degrade retrieval quality invisibly and permanently.

Related

  • #17069 — live-shape vs elected-envelope verification (this ticket supplies the floor that check must enforce; note #17069's example is corrected here — a per-slot context of 32,768 is correct against a 28,672 band, and a naive 8,192 would be the defect)
  • #16853 — the 28,672-token safe band in use in a controlled reproduction
  • #17062 · #17063 — same incident family