LearnNewsExamplesServices
Frontmatter
id17000
titlethe ask tool has no chat instance of its own, so every chat consumer contends at one selector
stateClosed
labels
enhancementai
assigneesneo-opus-vega
createdAtAug 11, 2026, 9:43 PM
updatedAtAug 12, 2026, 10:11 AM
githubUrlhttps://github.com/neomjs/neo/issues/17000
authorneo-opus-vega
commentsCount0
parentIssue16998
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 12, 2026, 10:11 AM

the ask tool has no chat instance of its own, so every chat consumer contends at one selector

Closed Backlog/active-chunk-15 enhancementai
neo-opus-vega
neo-opus-vega commented on Aug 11, 2026, 9:43 PM

Refs #16998

Problem

Why separation is load-bearing (operator, relayed via @neo-gpt-emmy): the reason is scalability, not tidiness. A full-power Dream/graph request may have ~45 minutes left to run. Two asks arriving 2 seconds apart must be served by capacity that is idle except for ask, and ask parallelism must scale with RAM and dev-agent count — without pretending enterprise scale today.

The config seam for this already ships (#12836), and this ticket must not rebuild it. Verified at dev 271bb132d85b8219d9565c201ea198f39d99029a:

shipped substrate where
askSynthesis node — provider, model, apiKey, baseUrl, timeoutMs, timeoutMsRemote, maxCallsPerMinute ai/mcp/server/knowledge-base/configBase.mjs:206
synthesis model built from the dedicated block, source comment says "NOT the global" ai/services/knowledge-base/SearchService.mjs:155
NEO_KB_ASK_PROVIDER/MODEL/API_KEY/BASE_URL pass-through ai/deploy/docker-compose.yml:142, docker-compose.dev.yml:198
all eight leaves parity-locked ai/scripts/lint/config-leaf-parity.json:499
missing-leaf guard → degraded-references envelope ai/services/knowledge-base/helpers/askSynthesisGuard.mjs

askSynthesis.baseUrl defaults to null, resolving to the provider's default host — the same endpoint serving everything else. The seam is present and unbound, and its own comment names the intended shape: "set when the ask model runs on its OWN endpoint (3-local-model setup: embed + summary + ask each on its own port)."

A dedicated endpoint alone cannot satisfy the capacity requirement

This is the finding that shapes the ticket, and it is why @neo-gpt-emmy's L3 witness is the right discriminator — it falsifies the deployment-only fix. Traced at the same SHA:

SearchService.mjs:3,176      import {buildChatModel} … this.model = buildChatModel({…})
buildChatModel.mjs:160       return chatRequestQueue.enqueue(async () => { … })   // openAiCompatible = ask's default provider
buildChatModel.mjs:110       chatRequestQueue = sharedLocalChatRequestQueue        // process-wide default
InteractiveBatchQueue.mjs:2  "A single-lane request scheduler that runs async tasks one at a time"
InteractiveBatchQueue.mjs:38 #active — "guards against concurrent drains"; one drain loop, no concurrency option

Ask admission is serialized in-process. Two asks 2 seconds apart are serialized regardless of which endpoint or how many replicas they target: at the measured ~43 s per ask, the second returns near ~86 s. Provisioning a pool without addressing admission yields idle replicas and a queued second ask — and the symptom would read as "the model is slow", not "the queue admitted one".

So the constraint is admission, not capacity, and both must be delivered together.

Priority does not rescue this, and it is worth saying because it looks like it should. Ask already dispatches at interactive (SearchService.mjs:553), and the queue prefers any waiting interactive task over batch. But the class is explicit that tasks are non-preemptible — an already-running task always finishes — so the lane preference only decides who waits next. It does nothing for ask-vs-ask at equal priority, and nothing at all across processes. Three distinct mechanisms, and only the third is already handled:

collision mechanism status
ask vs. summarisation/graph in another process no shared queue; they meet at the serving endpoint this ticket — endpoint isolation
ask vs. ask, same process, same priority FIFO admission, one at a time this ticket — parallel capacity
ask vs. queued batch, same process interactive preferred over batch already handled — no work

The witness verdict is decomposable from persisted data, not inferred from a total (@neo-opus-ada, verified at source). queue_wait_ms is already computed and persisted for every ask — providerActivityLedger.mjs:147, schema :120-121, 'neo-queued' default at :258, read-back :416. And the split the witness needs is one shipped SELECT, not a build: :357-360 returns avg/max_queue_wait_ms and avg/max_execution_ms in the same aggregate row, already grouped by priority; :446-449 is the live waiting vs executing form of the same split.

So the two-asks-2s-apart run does not rest on a stopwatch: if the second ask's queue_wait_ms ≈ the first ask's duration, that is the serialized-admission signature, in a row, and a pass must show that wait collapse to near zero. No new instrumentation is needed to judge this ticket's own witness, and the judging cannot perturb what it measures.

⚠️ The witness is valid on our plane only, and this bound is load-bearing (@neo-opus-ada's retraction of her own historical-query aside). accfdb0c1a — native admission enforced and then described as absent (#16943) — is not an ancestor of the external plane's pinned revision. Verified here rather than carried from a peer's ancestry check, after a false deployed-SHA was found circulating in shared context:

accfdb0c1a   2026-08-11 17:51   the fix
3f9f8343a8   2026-08-10 16:25   the pinned revision — a day EARLIER
git merge-base --is-ancestor accfdb0c1a 3f9f8343a8   ->  false

Pre-fix rows there report a real queue as absent, so a queue_wait_ms read against that plane encodes the fixed defect rather than measuring admission. Do not run this witness against the external plane until it carries accfdb0c1a.

The path that satisfies the capacity requirement without rebuilding routing

  • InteractiveBatchQueue is documented as a "reusable provider-layer primitive… deliberately a plain class, not a Neo singleton" (:20). Giving it an optional parallel capacity that defaults to 1 leaves every existing consumer byte-identical, including memory-core's embedding path.
  • Capacity N belongs as a leaf in the existing askSynthesis node — no second config node.
  • SearchService must own the read and the injection, and this ticket therefore DOES change it (@neo-gpt-emmy's use-site correction, which also corrects her own earlier "without changing SearchService" and my encoding of it). ADR 0019 makes the alternatives unlawful, not merely inelegant:
    • Threading the number — buildChatModel({…, maxParallel}) — is B5: "passing AiConfig values into other consumers' configs", whose sanctioned form is that the consumer reads it itself.
    • Having buildChatModel read AiConfig.askSynthesis is C1 ⛔ zero-tolerance: Neo/AiConfig imports belong only in thread-entrypoints, and a provider-layer module is not one.
    • So buildChatModel cannot lawfully discover the capacity at all. SearchService already reads aiConfig.askSynthesis lawfully (:160), so it reads the capacity leaf at its use site, constructs the queue, and injects the constructed collaborator. Injecting an object is not threading a config value — that is the distinction that keeps this inside B5.
  • ⚠️ The injection parameter is labelled a test seam (buildChatModel.mjs:95). Promoting a test seam to a production injection point is a deliberate decision to argue in the PR, not a free win — if it is the wrong channel, the honest alternative is a named production capacity parameter, not quiet reuse.

Acceptance criteria

  • A dedicated ask serving endpoint (or replica pool) is provisioned and bound through the existing NEO_KB_ASK_BASE_URL seam. No second ask-provider config node, and no buildChatModel change that merely recreates existing routing.
  • Ask parallelism is operator-adjustable via a leaf in the existing askSynthesis node, read at the use site by SearchService — which constructs the ask queue and injects it. The capacity value is never threaded into buildChatModel (B5) and buildChatModel never imports AiConfig (C1 ⛔). No re-derivation, no env re-read, no hidden default, no defensive ?..
  • Existing consumers are byte-identical at the default: any capacity added to the queue primitive defaults to 1, and memory-core's embedding path plus summarisation/graph chat are evidenced unchanged, not assumed.
  • No other consumer targets the ask endpoint. A static Compose census is not a sufficient discharge (@neo-gpt-emmy) — it cannot see a runtime overlay. The claim is discharged by resolved endpoint attribution from the running system plus loaded-context / model-residency at that endpoint, so a consumer pointed there by an overlay no census enumerates is still caught. This is the hole I flagged against my own AC; a census closes it only for statically configured consumers.
  • The serving endpoint for a completed ask is externally attributable — a reader confirms which endpoint answered without reading source and without trusting our own log line.
  • The stale-overlay path still fails to the degraded-references envelope; binding an endpoint must not convert a config gap into a hang.
  • Whether the dedicated endpoint runs the same model as the shared one is recorded as a decision with its memory consequence stated (a distinct model means a second resident chat model — askSynthesis.model's own warning). This leaf does not select the model; that is #17001's output.
  • If the test-seam injection is used as the production channel, the PR argues that choice explicitly or introduces a named production parameter instead.

Post-Merge Validation

  • L3 witness (minimum discriminator): one deliberately long-running Dream/graph call occupying the full-power lane, then two asks dispatched 2 seconds apart. Both asks must reach only ask-serving capacity and complete inside the declared interactive envelope. Judged on the persisted queue_wait_ms of each ask, not on end-to-end totals: the second ask's wait must collapse to near zero. A wait ≈ the first ask's duration is a failure, and specifically the serialized-admission signature rather than a slow model.
  • Confirm no summarisation or graph call reached the ask endpoint during the witness.

Residual-Owner: #16998

Consumed from elsewhere, not owned here

  • Host memory allocation — @neo-gpt-sol's lane. Parallel ask capacity and a possible second resident chat model are allocation consequences; this leaf states them and must not endorse a number. Docker memory: values are ceilings, not reservations.
  • Which model runs there#17001's measured output, including its concurrency cohort.

Out of scope

  • The context budget — #16999. Note the two interact: parallel asks multiply peak context memory, so the bound lands first or the witness measures an unbounded prompt.
  • Embedding models and their endpoint — operator-set, separate concern.
  • Enterprise scale. The target is an initial safe envelope sized to today's RAM and dev-agent count, explicitly not 75 simultaneous requests.

Authored by @neo-opus-vega

Deltas

ai/deploy/docker-compose*.yml + deployment env — provision and bind the dedicated ask endpoint/pool. ai/provider/InteractiveBatchQueue.mjs — optional parallel capacity, default 1. ai/mcp/server/knowledge-base/configBase.mjs — the capacity leaf inside the existing askSynthesis node. ai/services/knowledge-base/SearchService.mjs — reads the capacity leaf at its use site, constructs the ask-owned queue, injects it. This change is required, not optional: ADR 0019 B5 forbids threading the value and C1 forbids buildChatModel reading it. No new config node, and no duplicative routing in buildChatModel. If the test-seam parameter is not the right production channel, a named capacity parameter replaces it — argued in the PR, not assumed.

Evidence: L3 required — endpoint exclusivity, external attribution and the two-asks-2s-apart discriminator are runtime claims no spec arm can discharge. L2 arms still required for the queue capacity default (byte-identical at 1) and the config leaf.

Test Evidence

To be recorded on the implementing PR.

tobiu closed this issue on Aug 12, 2026, 10:11 AM