LearnNewsExamplesServices
Frontmatter
id12509
titleDesign V2 embedding QoS for add_memory contention
stateClosed
labels
enhancementaiarchitectureperformancemodel-experience
assigneesneo-gpt
createdAtJun 4, 2026, 4:13 PM
updatedAtJun 5, 2026, 4:00 PM
githubUrlhttps://github.com/neomjs/neo/issues/12509
authorneo-gpt
commentsCount0
parentIssue12487
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 5, 2026, 4:00 PM

Design V2 embedding QoS for add_memory contention

Closed v13.0.0/archive-v13-0-0-chunk-16 enhancementaiarchitectureperformancemodel-experience
neo-gpt
neo-gpt commented on Jun 4, 2026, 4:13 PM

Context

#12487's original problem statement called for priority/QoS so interactive add_memory does not starve behind heavy Knowledge Base sync embedding batches. PR #12502 ships the V1 resilience slice: contention-timeout retry, batch chunk/yield at the service layer, and an embedding write-canary. @neo-opus-grace approved that V1 and explicitly kept true QoS as a non-blocking follow-up.

The Problem

V1 makes interactive embedding more resilient, but it does not prove the operator-level invariant that interactive memory writes go first while batch maintenance is active. Retrying under contention can still mean interactive work waits behind batch work. The long-term shape needs a true QoS mechanism that either yields batch embedding to interactive requests or gives interactive embedding a priority lane.

Without a separate follow-up, #12487 would close on V1 and the deferred priority/QoS requirement would remain only in review prose.

The Architectural Reality

  • add_memory is a critical interactive write path because AGENTS.md requires every turn to persist memory.
  • KB sync / maintenance embedding is batch work and can be long-running.
  • The embedding provider can serialize requests to a single model, so service-level retries are a mitigation, not true scheduling authority.
  • PR #12502 intentionally avoids building a scheduler in V1. Its reviewer approved that scope split and named V2 true-QoS as the follow-up.

The Fix

Design and implement the V2 QoS mechanism for embedding contention. Evaluate at least these shapes before implementation:

  1. Batch-yield / chunk-with-gaps / pause-on-interactive: maintenance embedding yields when an interactive request is pending.
  2. Priority lane at the embed-request layer: interactive requests are queued ahead of batch requests.
  3. Orchestrator-level pause/resume: Knowledge Base sync or other batch embedding work pauses when Memory Core receives interactive write demand.

Choose the smallest substrate that proves the interactive add_memory invariant without turning the Memory Core service into a broad scheduler prematurely.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
Interactive embedding path (add_memory / TextEmbeddingService.embedText) #12487 operator invariant + PR #12502 V1 residual Interactive memory writes get priority or explicit yield over batch embedding Retain V1 retry if QoS cannot be activated PR body + service docs if API/config changes Contention test or live probe showing interactive request proceeds during batch
Batch embedding path (embedTexts / KB sync maintenance) #12487 + PR #12502 review Batch work yields or queues behind interactive demand without losing global ordering guarantees Existing chunk/yield V1 behavior Docs if new config/control surface is introduced Unit/integration coverage for yield/priority semantics and ordering
QoS control surface This ticket Explicit config or service primitive if needed; no hidden magic constants Defaults preserve V1 behavior when disabled Config template docs if added Config/contract tests for enabled and disabled states

Decision Record impact

none currently; if the chosen design creates a new scheduling primitive shared across services, file or amend a Decision Record before implementation.

Acceptance Criteria

  • V-B-A compares the three candidate QoS shapes and selects the smallest substrate that proves interactive priority/yield.
  • Interactive add_memory can complete while batch embedding is active without relying only on timeout retry.
  • Batch embedding preserves ordering and eventual completion when yielding or deprioritized.
  • Tests cover interactive-vs-batch contention with mocked providers; any live L3/L4 validation required is declared in the PR Evidence line.
  • PR body explicitly distinguishes V2 true QoS from PR #12502's V1 contention resilience.

Out of Scope

  • Reworking all orchestrator maintenance scheduling.
  • Multi-model provider coexistence (#12090 lineage).
  • Healthcheck canary TTL/frequency; tracked separately.

Related

Parent: #12487 Source PR: #12502 Related: #11393, #12090, #12450, #12508

Origin Session ID: dcdaac0b-9ae0-45b5-b4da-da39541af497

Handoff Retrieval Hints: query_raw_memories add_memory embedding contention priority lane QoS batch yield; PR #12502 review by @neo-opus-grace at head 6822412e88c56b3bb5904225f212283f04e9b77d.

tobiu closed this issue on Jun 5, 2026, 4:00 PM
tobiu referenced in commit e2a7dde - "feat(memory-core): prioritize interactive embeddings (#12509) (#12537) on Jun 5, 2026, 4:00 PM