LearnNewsExamplesServices
Frontmatter
id16932
titleThe ask tool defaults to a metered cloud model, and its local path was never wired — plus 31b is reachable from six places
stateClosed
labels
bugai
assigneesneo-opus-grace
createdAtAug 11, 2026, 12:24 AM
updatedAtAug 11, 2026, 9:01 AM
githubUrlhttps://github.com/neomjs/neo/issues/16932
authorneo-opus-grace
commentsCount2
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 11, 2026, 9:01 AM

The ask tool defaults to a metered cloud model, and its local path was never wired — plus 31b is reachable from six places

neo-opus-grace
neo-opus-grace commented on Aug 11, 2026, 12:24 AM

Operator directive, prio 0, issued 2026-08-11 after the Gemini API key was deleted: the ask tool must stop defaulting to a metered cloud model, and gemma-4-31b-it must be removed everywhere.

Two independent defects, one directive.

1. ask defaulted to a metered remote model

ai/mcp/server/knowledge-base/configBase.mjs declared:

provider: leaf('gemini',            'NEO_KB_ASK_PROVIDER', 'string'),
model   : leaf('gemini-2.5-flash',  'NEO_KB_ASK_MODEL',    'string'),

Every ask_knowledge_base call billed a cloud provider by default — roughly EUR 70/month steady-state — and required a dedicated key that a peer subsequently exposed. The key has now been deleted upstream, so on dev the ask path is not merely expensive, it is broken.

The block's existing cost-safety machinery — a 20/min runaway breaker, a dedicated env-only key, a per-path budget cap — all bounds a metered call. None of it removes the meter, and none of it can do anything about a leaked key.

2. The ask path could not have worked locally anyway

ai/services/knowledge-base/SearchService.mjs built its provider config as:

openAiCompatibleConfig: {...aiConfig.openAiCompatible, ...(ask.baseUrl ? {host: ask.baseUrl} : {}), model: ask.model}

Measured against the live KB config, {...aiConfig.openAiCompatible} is {}. An AiConfig node is a Neo.state.Provider proxy: its get trap walks the parent chain, but its ownKeys trap (Provider#getTopLevelDataKeys, src/state/Provider.mjs:676) enumerates local #dataConfigs only. These leaves live on the Tier-1 root, so from the Knowledge Base child every named read is correct and every enumeration is empty.

The spread silently dropped host, leaving OpenAiCompatible to fall back to its class default http://127.0.0.1:8000Chroma's port. A chat request aimed at a vector database.

This never surfaced because the default was gemini, which ignores that object entirely. Switching the default to local is precisely what would have exposed it, so the two defects had to be fixed together or the "fix" would have shipped a broken ask path.

3. gemma-4-31b-it was reachable from several places

26b was selected over 31b on measured performance grounds. LM Studio and MLX JIT-load whatever id they are handed, so a stray 31b string is not a naming inconsistency — it is a load instruction for a ~20 GB dense model that then sits resident beside the 15.6 GB one already serving traffic (observed live: both resident simultaneously, 31b holding a 60-minute idle TTL).

Consumable occurrences found: the MLX task leaf, both parity Compose profiles' NEO_KB_ASK_MODEL, the parity-latency diagnostics script, the mock server's chat-model fallback, and OpenAiCompatible's class default — the last being an Ollama-namespaced gemma4:31b inside the OpenAI-compatible provider, an id no OpenAI-compatible server can serve.

Acceptance Criteria

  • AC-1askSynthesis.provider defaults to openAiCompatible and askSynthesis.model to google/gemma-4-26b-a4b; remote synthesis remains available via NEO_KB_ASK_PROVIDER / NEO_KB_ASK_API_KEY.
  • AC-2 — the ask path resolves a host. SearchService reads the four leaves buildChatModel consumes at the use site and never materializes an AiConfig node.
  • AC-3NEO_KB_ASK_API_KEY (a Gemini credential) is never forwarded to a local OpenAI-compatible endpoint.
  • AC-4 — no gemma-4-31b-it / gemma4:31b / gemma4-31b remains in any consumable position under ai/ or test/. Explanatory prose that records why 26b was chosen, and dated historical benchmarks, are retained deliberately — deleting the rationale for a decision is not the same as removing the value.
  • AC-5 — a spec fails if the ask default regresses to a metered remote id, asserting on the model id and not only the provider name: a provider flip is visible in review, a model-id string edit is not.
  • AC-6 — the ask model id equals the Tier-1 openAiCompatible.model id, asserted across both files, so the two cannot drift into a second resident chat model.

Out of scope

The resources/content/archive/** corpus contains ~180 historical references to 31b inside archived issues, PRs and discussions. Those are records of what was said at the time and are deliberately untouched — rewriting them would falsify history rather than fix configuration.

The broader defect class this sits inside — 30 unit specs carrying test.skip(!!process.env.NEO_TEST_SKIP_CI, …), which skips in CI and therefore runs only against an operator's live LM Studio and Chroma — is tracked separately; it is what let a test dictate deployment model policy in the first place.

tobiu referenced in commit d6cba78 - "fix(knowledge-base): ask billed a cloud model by default, and its local path was never wired (#16932) (#16933) on Aug 11, 2026, 9:01 AM
tobiu closed this issue on Aug 11, 2026, 9:01 AM