LearnNewsExamplesServices
Frontmatter
id13283
titleP0: make add_memory acceptance independent of graph projection
stateClosed
labels
bugairegressionarchitectureperformancemodel-experience
assigneesneo-gpt
createdAtJun 15, 2026, 2:33 AM
updatedAtJun 15, 2026, 8:42 AM
githubUrlhttps://github.com/neomjs/neo/issues/13283
authorneo-gpt
commentsCount2
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtJun 15, 2026, 8:42 AM

P0: make add_memory acceptance independent of graph projection

Closed v13.1.0/archive-v13-1-0-chunk-3 bugairegressionarchitectureperformancemodel-experience
neo-gpt
neo-gpt commented on Jun 15, 2026, 2:33 AM

Authored by GPT-5.5 (Codex Desktop / Euclid).

Problem

add_memory is the mandatory end-of-turn durability gate. It must not fail, stall, or wait on derived projection work once the turn payload has been durably accepted.

Current source verifies that the Chroma embed has already been moved off the hot path, but the SQLite graph projection is still synchronous and can still veto the tool result:

  • ai/services/memory-core/MemoryService.mjs:399-407 appends the full payload to the durable JSONL WAL first.
  • ai/services/memory-core/MemoryService.mjs:409-438 then synchronously writes the AGENT_MEMORY graph node and graph edges before returning.
  • ai/services/memory-core/MemoryService.mjs:451-469 runs inline miniSummary enrichment as fire-and-forget, which is acceptable for the write path.
  • ai/services/memory-core/MemoryService.mjs:477-485 still returns MEMORY_ADD_ERROR for local write failures, and the comment names SQLite graph alongside WAL filesystem failures.
  • test/playwright/unit/ai/services/memory-core/MemoryService.WriteAhead.spec.mjs:23-34 pins the current contract as "never fail on content store" while still requiring synchronous graph recency.

That leaves the highest-value invariant only partially implemented: Chroma can no longer block the turn save, but graph projection still can.

Live Friction Evidence

During a live maintenance window on 2026-06-15, the orchestrator was running a Knowledge Base sync and deferring memory miniSummary backfill:

[Orchestrator] Deferring memory miniSummary backfill; heavy maintenance task knowledge base sync is active (pending-memory-minisummary:3647).

Source confirms this is KB heavy-maintenance backpressure, not graph maintenance:

  • ai/daemons/orchestrator/services/MaintenanceBackpressureService.mjs:18-26 classifies both memory-summary-backfill and kbSync as exclusive heavy-maintenance tasks.
  • ai/daemons/orchestrator/taskDefinitions.mjs:97-112 defines memory-summary-backfill and kbSync as separate task lanes.
  • ai/daemons/orchestrator/services/MaintenanceBackpressureService.mjs:192-197 emits the observed deferral line.

This ticket does not claim KB sync is graph work. The evidence matters because it shows the system already treats Memory Core derivations as maintenance/backlog work, while the mandatory add_memory acceptance path still contains a synchronous graph projection step.

Architectural Direction

Make add_memory a WAL-acceptance tool:

  1. Validate payload and append to the durable WAL.
  2. Return success once the WAL append succeeds.
  3. Move graph node/edge projection, miniSummary enrichment, and Chroma embedding into asynchronous, retryable derived processors.
  4. Preserve read-after-write ergonomics through an explicit pending overlay or projection status rather than making graph writes part of the acceptance gate.

This must honor both deployment shapes already used by the WAL embed drain:

  • Local/orchestrator profile: supervised daemon process is acceptable.
  • Cloud/container profile: in-process hosted loop is required when there is no local orchestrator process.

The existing embed drain design is the closest implementation pattern: ai/daemons/embed/drainCycle.mjs:245-253 explicitly supports both supervised daemon and in-process Memory Core hosting, with a sole-drainer lock. A graph-projection drain should follow the same deployment-aware shape instead of assuming a local desktop daemon exists.

Acceptance Criteria

  • add_memory returns success after a valid payload is durably appended to the WAL, even when graph projection is unavailable, contended, or temporarily failing.
  • A WAL append failure or validation failure may still reject the save; graph/miniSummary/Chroma failures must not reject an otherwise accepted turn.
  • Graph projection is retried asynchronously with bounded backoff and observable status; no accepted turn is silently dropped.
  • query_recent_turns and related recency surfaces either read pending WAL entries directly or expose an explicit projection-pending state, so callers do not mistake eventual graph projection for data loss.
  • Cloud deployments are first-class: no design may require an external local orchestrator daemon as the only projection host.
  • Unit coverage includes a falsifier where graph projection throws or is unavailable and add_memory still succeeds with the WAL record pending.
  • Existing coverage that proves Chroma is not touched by add_memory remains intact.
  • Operator/debug health surfaces distinguish accepted-to-WAL, graph-projected, miniSummary-ready, and embedded-to-Chroma states.

Out Of Scope

  • Reverting the WAL embed daemon.
  • Treating KB sync as graph maintenance.
  • Requiring immediate Chroma semantic availability after add_memory.
  • Changing mailbox semantics beyond any existing piggyback metadata on the add_memory response.

Related Context

  • Prior WAL decoupling lineage: #12838 / #12840.
  • Recency surface lineage: query_recent_turns relies today on synchronously-written AGENT_MEMORY graph rows.
  • Current heavy-maintenance source separates KB sync from memory miniSummary backfill, but both share the exclusive heavy-maintenance class.

Release Integration Classification

P0 Agent OS reliability regression / Memory Core durability contract. This is not a cosmetic performance item: add_memory is the mandatory turn-save gate, so any non-WAL dependency in its success path risks permanent turn loss or blocked lifecycle closure.