LearnNewsExamplesServices
Frontmatter
id16768
titleask has no reasoning-effort control, so it spends its budget thinking
stateClosed
labels
bugaiperformanceagent-os
assigneesneo-opus-grace
createdAtAug 9, 2026, 3:52 AM
updatedAtAug 9, 2026, 12:21 PM
githubUrlhttps://github.com/neomjs/neo/issues/16768
authorneo-opus-grace
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 9, 2026, 12:21 PM

ask has no reasoning-effort control, so it spends its budget thinking

Closed Backlog/active-chunk-14 bugaiperformanceagent-os
neo-opus-grace
neo-opus-grace commented on Aug 9, 2026, 3:52 AM

Context

Measured by @neo-gpt on 2026-08-09 (M5 Max, Neo's own OpenAiCompatible adapter, Gemma 4 26B, a 7,890-token grounded prompt):

run latency outcome
reasoning_effort: none 33.3 s valid answer
no control sent (current behavior) 86.7 s empty final answer — 297 of 299 completion tokens spent reasoning

The second row is the defect. It is not merely 2.6× slower: the model consumes its entire completion budget on reasoning tokens and returns nothing, so the caller receives a successful, empty response after 87 seconds.

Evidence handed over on the Neo plane with an explicit invitation to push back; the seam below was independently verified at dev before filing.

The Problem

Neo already has this control, wires it for two consumers, and omits the third.

ai/configBase.mjs:756-761:

 * `SessionService.summarizeSession` reads `summaryReasoningEffort`;
 * `SemanticGraphExtractor.executeTriVectorExtraction` reads `graphReasoningEffort`.
summaryReasoningEffort: leaf('none', 'NEO_LOCAL_MODELS_CHAT_SUMMARY_REASONING_EFFORT', 'string'),
graphReasoningEffort  : leaf('none', 'NEO_LOCAL_MODELS_CHAT_GRAPH_REASONING_EFFORT', 'string')

Both existing leaves default to 'none'. Three consumers send the value today — SessionService.mjs:711, SemanticGraphExtractor.mjs:593, TopologyInferenceEngine.mjs:282. The ask path sends nothing:

  • askSynthesis's required-leaf set is ['provider', 'model', 'timeoutMs', 'timeoutMsRemote', 'maxCallsPerMinute'] (ai/services/knowledge-base/SearchService.mjs:92) — no reasoning-effort member.
  • SearchService.ask's generateContent call (:477) passes no reasoning_effort.

So the asymmetry is exact and unintentional-looking: summary and graph get no-think by default; interactive ask — the one path a human waits on — reasons by default. #13853 / #13854 introduced the control for summary and graph; no ask-specific successor exists.

Mini-summary shares the omission: it does not send the summary effort either.

The Architectural Reality

  • ai/configBase.mjs:751 already documents the value as "passed straight through as the OpenAI / LM-Studio reasoning_effort", so the transport contract is settled and both local providers accept it.
  • ai/services/knowledge-base/SearchService.mjs:92 the guard's required-leaf list, :477 the generateContent call that would carry it.
  • ai/services/knowledge-base/helpers/askSynthesisGuard.mjsgetMissingAskSynthesisLeaves, which fails closed with a remediation message naming --migrate-config. A new required leaf must be added here coherently or a stale overlay degrades with a misleading reason.
  • The two sibling leaves establish the provider-neutral value and default; the ask-owned block establishes placement and the NEO_KB_ASK_* env convention.

This is a one-leaf change with a shipped template, not a design question.

The Fix

  1. Add reasoningEffort: leaf('none', 'NEO_KB_ASK_REASONING_EFFORT', 'string') to ai/mcp/server/knowledge-base/configBase.mjs under askSynthesis, where the ask provider, model, and endpoint are owned.
  2. Read AiConfig.askSynthesis.reasoningEffort at the use site in SearchService.ask and pass it as reasoning_effort on the generateContent call, matching SessionService.mjs:711's || undefined idiom so an empty value omits the field rather than sending an empty string.
  3. Decide whether the leaf joins askSynthesisGuard's required set. Adding it there makes a stale overlay fail closed with a clear remediation; leaving it out lets an old overlay keep working with today's behavior. Recommendation: not required — a missing effort leaf should degrade to current behavior, not refuse to answer, because refusing is worse than answering slowly.
  4. Apply the same treatment to mini-summary, which omits the summary effort.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback / Error Semantics Docs Evidence
AiConfig.askSynthesis.reasoningEffort ai/mcp/server/knowledge-base/configBase.mjs, per ADR-0019 Declarative leaf, default 'none', env NEO_KB_ASK_REASONING_EFFORT Absent/empty ⇒ field omitted from the request, i.e. exactly today's behavior KB search guide Canonical leaf resolves through the KB config template and is registered in config-leaf parity
SearchService.askgenerateContent existing call site :477 Sends reasoning_effort read at the use site || undefined so an empty value omits the field, per SessionService.mjs:711 A grounded ask returns a non-empty answer within the timeout
askSynthesisGuard required-leaf set SearchService.mjs:92 unchanged — the new leaf is optional A stale overlay keeps working rather than failing closed remediation string Negative spec: an overlay without the leaf still answers
mini-summary effort existing summary leaf Sends summaryReasoningEffort like SessionService does same || undefined idiom Mini-summary request carries the field

Decision Record impact

aligned-with ADR 0019 — a declarative leaf(default, env, type) read at the use site, with two shipped siblings as precedent. No new resolution path, no formula, no threading, no runtime mutation. Nothing amended.

Acceptance Criteria

  • AiConfig.askSynthesis.reasoningEffort exists as a canonical leaf with default 'none', env NEO_KB_ASK_REASONING_EFFORT, and is registered in config-leaf parity.
  • SearchService.ask sends reasoning_effort read at the use site; an empty value omits the field rather than sending an empty string.
  • [L3-deferred — operator handoff needed] The defect is reproduced before the fix and gone after: a grounded ask against a reasoning-capable local model returns a non-empty answer, where the same prompt previously exhausted its completion budget on reasoning tokens and returned empty.
  • Negative control: an overlay lacking the leaf still answers — the guard does not gain a new required member, so a stale config degrades to today's behavior rather than refusing.
  • Mini-summary sends the summary effort.
  • No consumer reads the value anywhere but its use site, and no default is shadowed service-side.

Out of Scope

  • Choosing a different ask model or endpoint. @neo-gpt's stronger hypothesis for interactive ask — a smaller dedicated model rather than sharing a 26B with summary and graph — is a separate and larger question. This ticket makes the existing model usable; it does not pick a new one.
  • Embedding parallelism tuning (1 vs 4 lanes, shared vs isolated endpoints). Deployment-side, and per @tobiu it belongs after ingestion works.
  • Why an external plane's corpus is empty#16706 and #14154.
  • Any change to the two existing leaves or their consumers.

Avoided Traps

  • Reading the 2.6× latency as the defect. The latency is the visible half; the empty answer is the failure. A fix validated only on wall-clock would pass while still returning nothing on a longer prompt.
  • Adding the leaf to the guard's required set because that feels stricter. It would convert a slow answer into no answer for every existing overlay — strictly worse for the user this ticket exists to help.
  • Assuming the transport needs work. It does not: three consumers already send reasoning_effort through the same adapter, and configBase.mjs:751 documents the pass-through. This is an omitted consumer, not a missing capability.
  • Benchmark-cohort mixing. The measurements here are M5 Max; an EPYC deployment is an independent cohort (@tobiu). A regression check must state which cohort it ran on.

Related

#13853 / #13854 (introduced the control for summary + graph; this is the omitted third consumer) · #16706 (an external plane where ask is separately unusable on an empty corpus — two independent failure modes) · #14154 (the eviction candidate for that empty corpus) · ADR-0019

Live latest-open sweep: checked latest 20 open issues created-descending at 2026-08-09T01:52Z; #16766 and #16767 are the newest and neither overlaps. A2A in-flight sweep: 30 most recent across all read-states; @neo-gpt's handover explicitly states "no repo, ticket, or deployment mutation; no lane claim", so this lane is unclaimed.

Origin Session ID: a641ddac-565a-4fc8-adc1-6c25629bddb7

Retrieval Hint: query_raw_memories("ask reasoning_effort empty answer completion budget spent reasoning askSynthesis leaf") · configBase.mjs:760 · SearchService.mjs:477

tobiu referenced in commit 5927ad4 - "feat(ai): ask sends a reasoning-effort control instead of thinking its budget away (#16768) (#16772) on Aug 9, 2026, 12:21 PM
tobiu closed this issue on Aug 9, 2026, 12:21 PM