LearnNewsExamplesServices
Frontmatter
id17056
titleFilesystem sync rewrites unchanged CONTAINS edges into GraphLog
stateClosed
labels
bugaiperformanceai-generatedagent-os
assigneesneo-gpt-emmy
createdAtAug 13, 2026, 4:04 PM
updatedAtAug 14, 2026, 1:54 AM
githubUrlhttps://github.com/neomjs/neo/issues/17056
authorneo-gpt-emmy
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 14, 2026, 1:54 AM

Filesystem sync rewrites unchanged CONTAINS edges into GraphLog

Closed Backlog/active-chunk-15 bugaiperformanceai-generatedagent-os
neo-gpt-emmy
neo-gpt-emmy commented on Aug 13, 2026, 4:04 PM

Context

The 2026-08-13 Memory Core liveness incident in #16677 exposed a producer-side amplification mechanism after the consumer wedge was isolated. A frozen GraphLog window contained 84,794 raw rows for 56,768 distinct graph entities; the dominant waves were filesystem CONTAINS edges that already existed in the authoritative graph.

One measured post-restart burst contained 27,642 CONTAINS edge updates. The wake consumer must remain bounded regardless of producer behavior, but replaying an unchanged repository tree should not manufacture tens of thousands of fresh invalidations.

The Problem

FileSystemIngestor.walkDirectory() correctly skips unchanged file/directory node writes using mtime/hash state, but it still calls GraphService.linkNodes(parentId, nodeId, 'CONTAINS', 1.0) for every traversed path.

GraphService.linkNodes() is a reinforcing relation operation: when the edge exists it increases its weight and executes UPDATE Edges. SQLite's edge-update trigger appends a GraphLog row. Filesystem topology verification is therefore accidentally expressed through a reinforcement primitive; every REM filesystem pass mutates unchanged topology and wakes every downstream invalidation consumer.

This is write amplification, not evidence that files, graph entities, or KB documents were erased.

The Architectural Reality

  • ai/services/memory-core/FileSystemIngestor.mjs:274-345 owns deterministic repository-tree projection. Its node path is already differential, while its CONTAINS path is not.
  • ai/services/memory-core/GraphService.mjs:537-603 owns linkNodes(). Existing callers may rely on its reinforcing-weight semantics; changing that method globally would silently alter unrelated graph learning.
  • ai/graph/storage/SQLite.mjs owns the edge-update trigger that records real mutations in GraphLog. Suppressing the trigger would hide legitimate updates from all producers and is the wrong layer.
  • #16677 bounds GraphLog consumption. This ticket reduces the verified producer burst; neither repair substitutes for the other.
  • Agent OS structure-map owner: ai/services/memory-core (FileSystemIngestor.mjs and GraphService.mjs are existing sibling owners; no new directory is required).

The Fix

  1. Add or expose an idempotent GraphService structural-edge operation that creates the requested relation when absent and otherwise leaves an equivalent edge unchanged.
  2. Switch only FileSystemIngestor's hierarchical CONTAINS projection to that operation.
  3. Preserve linkNodes()'s reinforcing semantics for consumers that intentionally accumulate relation weight.
  4. Report created versus verified edges truthfully in filesystem-sync statistics/logging.
  5. Prove the second unchanged sync performs zero CONTAINS edge updates and appends zero corresponding GraphLog rows.

Contract Ledger

Target surface Source of authority Proposed behavior Fallback / failure posture Evidence
Filesystem CONTAINS projection FileSystemIngestor.walkDirectory() Equivalent parent-child topology is verify-only Missing endpoints still cull/refuse as today Two-pass filesystem fixture
Structural edge operation GraphService Create when absent; no update when source, target, type, and structural properties are already equivalent Unexpected property drift is explicit, never silently reinforced Unit matrix: missing / equivalent / changed
Existing linkNodes() GraphService Retains reinforcing-weight semantics No global semantic change Positive control for an intentional reinforcing caller
GraphLog SQLite edge triggers Records actual create/update/delete mutations only No trigger suppression Row-count assertions around two syncs
FileSystemIngestor.syncWorkspaceToGraph({rootDir}) receipt FileSystemIngestor production entrypoint; default root remains the Neo checkout Returns {status, pathNodesUpserted, edgesCreated, edgesVerified, edgesDrifted, edgesCulled, edgesUnavailable} while existing Dream/restore callers may continue discarding it Graph storage unavailable returns the same shaped receipt with status: unavailable; injected root is fixture-only Production-entrypoint first / unchanged / add matrix
GraphService.ensureStructuralEdge() transaction boundary GraphService owns the structural mutation transaction Refuses an already-open Graph Database transaction; unlike linkNodes(), it does not consume a partial queued node/edge overlay Throws before any mutation when an outer transaction is active Outer-transaction zero-write falsifier

Decision Record impact

None. This restores producer idempotency inside the existing Native Edge Graph projection boundary.

Acceptance Criteria

  • A first filesystem sync creates the expected nodes and CONTAINS edges.
  • A second sync over the identical tree performs zero edge updates, preserves edge weights, and appends zero CONTAINS invalidations to GraphLog.
  • Adding one path produces only its required topology mutation; unchanged siblings stay byte-stable. Filesystem move/deletion reconciliation remains outside this leaf.
  • Existing intentional linkNodes() reinforcement remains covered and unchanged.
  • Filesystem-sync receipts distinguish created, verified, drifted, and culled topology without conflating verification with mutation.
  • The regression uses a real SQLite-backed graph fixture, not a mocked GraphLog count.

Out of Scope

  • Replacing GraphLog or the wake-subscription architecture.
  • Bounding the consumer drain; #16677 owns that independent liveness invariant.
  • General REM scheduling, provider budgeting, or graph compaction.
  • Filesystem deletion/reconciliation semantics beyond the existing contract.

Avoided Traps

  • Make linkNodes() globally idempotent. Other graph producers may intentionally reinforce weights; the filesystem needs a different operation.
  • Suppress SQLite edge-update triggers. That would hide legitimate mutations from wake and recovery consumers.
  • Treat existing edges as corpus loss. The measured authoritative rows remained present; the defect is repeated mutation.

Duplicate and Collision Sweep

  • Live latest-open sweep at 2026-08-13T14:04Z: checked the newest 20 open issues plus exact open/all-state searches for FileSystemIngestor, CONTAINS, GraphLog amplification, and filesystem rewrites; no equivalent open owner found.
  • Authoritative recent-message sweep at 2026-08-13T14:04Z: the MCP surface was unhealthy, so the same graph store was opened read-only and the latest 30 MESSAGE nodes were checked across all read states; only this author's earlier #16677 lane claim names the producer scope.
  • Merged PR #9943 fixed duplicate edge identities by adopting GraphService.linkNodes(); reinforcing verification remained.
  • Merged PR #9936 established that edge verification must remain outside the unchanged-node gate so missing topology self-heals.
  • Merged PR #15757 contains the boot-seed create-if-absent precedent; it does not expose a filesystem structural-edge contract.
  • Open PR #17058 / #16677 bounds the consumer drain and explicitly leaves this producer repair independent.
  • #12329 owns retained GraphLog compaction; it does not make live filesystem projection idempotent.
  • #17046 owns workload budgeting; it does not suppress unchanged graph writes.

Related

Related: #16677

Related: #12329

Related: #17046

Origin Session ID: ec35ab33-684f-40a9-804b-83fc32b21ac1

Retrieval Hint: "FileSystemIngestor unchanged CONTAINS edge GraphLog amplification REM"

— Emmy (GPT-5.6 Sol Ultra, Codex) 🪡

tobiu referenced in commit b810a1b - "feat(ai): make filesystem structural edges write-idempotent (#17056) (#17061) on Aug 14, 2026, 1:54 AM
tobiu closed this issue on Aug 14, 2026, 1:54 AM