LearnNewsExamplesServices
Frontmatter
id17155
titleThe embedding oversize guard disables itself on any unrecognised provider, and the provider name has no domain to recognise
stateClosed
labels
bugaiarchitecture
assigneesneo-kimi-phoebe
createdAtAug 15, 2026, 10:35 AM
updatedAtAug 15, 2026, 4:31 PM
githubUrlhttps://github.com/neomjs/neo/issues/17155
authorneo-opus-grace
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 15, 2026, 4:31 PM

The embedding oversize guard disables itself on any unrecognised provider, and the provider name has no domain to recognise

Closed Backlog/active-chunk-16 bugaiarchitecture
neo-opus-grace
neo-opus-grace commented on Aug 15, 2026, 10:35 AM

Problem

The oversize-input guard that keeps a document under the embedding context band is conditional on the provider name, and it fails open.

IngestionService.mjs:74 defines the entire enabled set:

const LOCAL_EMBEDDING_PROVIDERS = new Set(['openAiCompatible', 'ollama']);

resolveEmbeddingInputGuardrail() returns enabled: LOCAL_EMBEDDING_PROVIDERS.has(embeddingProvider), and when that is false:

  • IngestionService.filterEmbeddingInputBudget() returns every chunk unfiltered, and
  • VectorService.measureEmbeddingInput() returns {skip: false, inputBytes: 0, inputTokensEstimate: 0}without measuring anything.

So on an unrecognised provider a 100,000-token document is sent whole, and the two zeros it reports downstream are indistinguishable from a genuinely tiny input. The guard does not merely stop protecting; it produces a confident-looking measurement it never took.

This is reachable, not theoretical. gemini is a fully implemented embedding provider (TextEmbeddingService.mjs:310, 532, 711, 1993), and the selector carries no domain at all:

embeddingProvider: leaf('openAiCompatible', 'NEO_EMBEDDING_PROVIDER', 'string')

'string' accepts anything. NEO_EMBEDDING_PROVIDER=gemini disables the guard; so does NEO_EMBEDDING_PROVIDER=openaicompatible, or any typo, silently and with no diagnostic.

The inversion is the sharp part: the providers where the guard is off are the ones whose input limits are tighter than our 28,672-token band, not looser. A guard scoped to "local" providers protects exactly the deployments that need it least.

The Architectural Reality

  • #17070 closed the sibling hole — a provider that truncates silently — by minting EMBEDDING_INPUT_TRUNCATED from either Neo's bounded LM Studio estimate or an OpenAI-compatible provider's structured exceed_context_size_error. That typed handling is scoped to the same two provider shapes as the pre-check, so a third provider is uncovered on both the pre-check and the refusal-classification paths.
  • The 'string' domain is the same defect class @neo-gpt caught in backoffBaseMs on #17126 today: a loose type where the value is an enum. There, 'number' accepted a negative delay; here, 'string' accepts a provider that does not exist.
  • The band itself (EMBEDDING_SAFE_PROCESSING_LIMIT_TOKENS = 28672) and the splitter are sound and mutually consistent — the splitter budgets safeProcessingLimitTokens * 3 bytes, the exact inverse of bytesToTokens. Nothing below is about the band being wrong; it is about the band not being consulted.

The Fix

  1. Fail closed on an unrecognised provider. Measure regardless, and treat "provider not classified" as a refusal to send rather than a licence to send unmeasured. An unknown provider is precisely the case where we know least about the limit.
  2. Give embeddingProvider a domain. The set of implemented providers is knowable at config time; a typo should fail loudly at boot rather than silently disable a safety guard at ingestion.
  3. Stop reporting zeros for an unmeasured input. {inputBytes: 0, inputTokensEstimate: 0} is a measurement claim. An unmeasured input needs a distinguishable shape so diagnostics cannot read "not checked" as "checked, tiny".

Out of Scope

  • Recalibrating BYTES_PER_TOKEN_HEURISTIC. That axis was measured against a live tokenizer on 2026-06-23 (dense agent prose at 2.72 chars/token, code at ~4.0 — code is not the dense case) and mitigated from 4 to 3. The residual margin question belongs with that measurement, not here.
  • Changing the band, the splitter, or the drop-with-receipt behaviour. Those work: oversized documents are split and any residue is dropped, never truncated — TextEmbeddingService.mjs:98, "A truncated embedding must never be stored."

Acceptance Criteria

  • An embedding provider outside the recognised set produces a refusal to send an over-band input, not an unfiltered pass, at both the ingestion filter and the VectorService boundary.
  • An unmeasured input is distinguishable from a measured-and-tiny one at every consumer of measureEmbeddingInput, with a control asserting the two shapes differ.
  • embeddingProvider carries a domain over the implemented providers; an unknown value fails at config resolution with a named diagnostic rather than silently disabling the guard.
  • A falsifier drives a 100,000-token document through a non-local provider name and asserts nothing over the band reaches the provider — the case today's tree passes by not looking.
  • The gemini path specifically is covered by whichever branch the fix chooses (guarded, or explicitly refused as unsupported for ingestion), since it is implemented and reachable today.

Evidence class

Source-verified 2026-08-15 at dev: IngestionService.mjs:74 provider set, :1400-1406 guardrail resolution, :1433-1486 unfiltered return, VectorService.mjs:744-756 unmeasured zeros, configBase.mjs:722 'string' selector, TextEmbeddingService.mjs gemini branches. Prior art read and deliberately not duplicated: #17070 (closed, typed overflow), #13929 (closed, the local-provider guard), #13918 (closed, heuristic calibration), #17113 (open, slot-fit admission).

Authored by Grace (Claude Opus 5, Claude Code). Session b17338dd-b474-494f-b08c-683044de2ddb.

tobiu referenced in commit b027c0c - "feat(ai): charge the review budget per reviewer family, and make the exception checkable (#17163) (#17164) on Aug 15, 2026, 2:53 PM
tobiu referenced in commit 5c2c263 - "fix(ai): fail the embedding oversize guard closed on unrecognized providers and give the provider selector a domain (#17155) (#17167) on Aug 15, 2026, 4:31 PM
tobiu closed this issue on Aug 15, 2026, 4:31 PM