LearnNewsExamplesServices
Frontmatter
id10230
titleupsertNode must lazy-load from SQLite before cache decision
stateClosed
labels
bugaiarchitecturecore
assigneesneo-gemini-pro
createdAtApr 23, 2026, 2:00 PM
updatedAtApr 23, 2026, 3:46 PM
githubUrlhttps://github.com/neomjs/neo/issues/10230
authorneo-opus-ada
commentsCount0
parentIssue10186
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtApr 23, 2026, 3:46 PM

upsertNode must lazy-load from SQLite before cache decision

Closed v13.0.0/archive-v13-0-0-chunk-5 bugaiarchitecturecore
neo-opus-ada
neo-opus-ada commented on Apr 23, 2026, 2:00 PM

Context

During the 2026-04-23 session-long diagnostic into AgentIdentity wipes that drove PRs #10220, #10221, #10225, #10227, #10229 (sessions 8968b9f6, 24aa1fa1), we traced three distinct wipe/corruption mechanisms. PR #10229's apoptosis-exemption fix closed one; two remain. This ticket fixes the architectural root-cause enabler of the second.

The Problem

GraphService.upsertNode:110-163 never lazy-loads from SQLite before deciding create-vs-update:

upsertNode({id, type, name, ...}) {
    let node = this.db.nodes.get(id);  // in-memory Map lookup only
    if (node) { /* merge */ } else { /* addNode → creates fresh → overwrites SQLite */ }
}

If a rich-metadata node exists in SQLite (e.g., seeded AgentIdentity with githubLogin/displayName/modelFamily/accountType/createdAt) but hasn't been pulled into the in-memory cache yet (fresh boot, node not yet queried), upsertNode takes the "create fresh" branch. Storage writes via ON CONFLICT(id) DO UPDATE SET data=excluded.data OVERWRITE the existing rich row with whatever minimal stub the caller provided.

This silently destroys seed metadata when any code path calls upsertNode with a partial payload from a cold-cache state — the core enabler of the PermissionService lazy-stub hazard (filed as sibling ticket).

Empirical evidence: session 8968b9f6 observed AGENT:* with type 'AGENT' (the lazy-stub shape) instead of 'BroadcastSentinel' (the seed shape from #10174). Attribution: some code path called upsertNode-with-stub-payload before the seeded node was cache-loaded; the stub's JSON overwrote the rich seed JSON in SQLite.

The Architectural Reality

  • ai/mcp/server/memory-core/services/GraphService.mjs:110-163 — upsertNode implementation (cache-only check)
  • ai/mcp/server/memory-core/services/GraphService.mjs:422-425 — getNode establishes the precedent: call this.db.getAdjacentNodes(id, 'both') to trigger lazy-load BEFORE the in-memory check
  • ai/graph/storage/SQLite.mjs:117-132 — addNodes writes ON CONFLICT DO UPDATE SET data=excluded.data (full row replacement, no JSON merge)

Memory Core search (query_raw_memories("upsertNode lazy-load cache SQLite decision")) confirmed NO documented design rationale for the cache-only check pattern. This is a latent architectural gap, not a deliberate design choice.

The Fix

Prepend a lazy-vicinity-load to upsertNode, mirroring the getNode pattern:

upsertNode({id, type, name, description, semanticVectorId, state, updatedAt, properties}) {
    // Lazy-load from SQLite before in-memory check — prevents cold-cache stubs from
    // overwriting rich SQLite rows via addNodes' ON CONFLICT DO UPDATE semantics.
    // Mirrors the discipline in getNode:424. See ticket body for empirical motivation.
    this.db.getAdjacentNodes(id, 'both');

    let node = this.db.nodes.get(id);
    // ... existing logic unchanged
}

Acceptance Criteria

  • upsertNode first call with an ID that exists in SQLite but not in cache triggers lazy-load → in-memory check returns the full node → update-branch taken → properties merged (not overwritten)
  • Regression test: seed @test-identity directly via SQLite addNodes (bypassing upsertNode), clear db.nodes.clear() to simulate cold cache, call upsertNode({id: '@test-identity', type: 'AGENT'}) — assert the original properties (e.g., githubLogin) are preserved and type reflects the new value
  • Existing upsertNode tests continue to pass unchanged
  • Anchor & Echo inline comment at the new lazy-load line citing this ticket + the root-cause narrative

Out of Scope

  • Changes to storage.addNodes JSON-merge semantics (would require a broader schema-migration design)
  • Changes to linkNodes — already does SQLite-direct existence check, unaffected
  • Refactoring to unify upsertNode/getNode/linkNodes into a common cache-discipline primitive (larger scope; possible future work)

Avoided Traps

  • "Use JSON-merge at SQLite layer instead": rejected. SQLite JSON merge would require schema-version awareness and much larger scope. The lazy-load fix is ~2 lines and addresses the root cause directly.
  • "Let upsertNode take a shallow-overwrite flag": rejected. Adds a flag-surface callers will forget; lazy-load-before-decision is invariant-correct for ALL callers.
  • "Just document that callers must pre-load the cache": rejected. Shifts invariant enforcement to every caller; guaranteed to drift. Invariants belong at the bottleneck.

Related

  • Discovered by: sessions 8968b9f6 + 24aa1fa1 during post-merge A2A validation (2026-04-23)
  • Enables latent hazard behind: PermissionService.grantPermission:65-68 lazy-stub (filed as sibling)
  • Adjacent: #10190 (cache coherence via syncCache delta replay) — this fix extends the coherence discipline to the write path
  • Pattern precedent: getNode:422-425 already implements lazy-load-before-cache-check correctly
  • Parent epic (soft): #10186 (MCP concurrency audit) — graph-write discipline is part of the concurrency substrate

Origin Session ID: 24aa1fa1-9a22-498e-97f2-760c12e5a79d

Handoff Retrieval Hints

  • query_raw_memories(query="upsertNode lazy-load cache SQLite decision")
  • query_raw_memories(query="AgentIdentity wipe partial state overwrite")
  • query_raw_memories(query="AGENT:* BroadcastSentinel type overwritten AGENT")
tobiu referenced in commit e616670 - "fix(memory-core): force upsertNode SQLite lazy-load on cache miss (#10230) (#10234) on Apr 23, 2026, 3:46 PM
tobiu closed this issue on Apr 23, 2026, 3:46 PM