Context
Post-#10239 restart validation (session 22f662c4, 2026-04-23 14:02 UTC) empirically confirmed a boot-time race that the four merged wipe-mechanism defenses (#10229, #10232, #10234, #10235) do not close. The race is upstream of the graph substrate but downstream of the identity resolver. #10176's new healthcheck.identity block provides the definitive signal for the first time.
This ticket is specifically for a fresh session to deep-dive forensics. All prior diagnostic state is captured below so the next agent doesn't re-derive it.
The Problem
Observed state (session 22f662c4, post-#10239 restart)
healthcheck → {
"identity": {
"source": "env-var",
"bound": false,
"nodeId": null
}
}But at the same session 61 seconds later:
get_node({id: "@neo-opus-ada"}) → {
"id": "@neo-opus-ada",
"type": "AgentIdentity",
"name": "Claude Opus 4.7",
"description": "Anthropic Claude Opus version 4.7 Agent Identity"
}
get_node({id: "AGENT:*"}) → {
"id": "AGENT:*",
"type": "BroadcastSentinel",
...
}The nodes exist with correct types and full seed metadata. bindAgentIdentity couldn't find them at boot. The graph query later finds them. This is either a boot-time timing race OR a silent-rejection pattern in the init chain.
What does NOT explain it
- ❌ NEO_AGENT_IDENTITY missing from spawn env — ruled out,
source: 'env-var' confirms propagation
- ❌
gh-CLI timeout — ruled out, not in resolver chain when env-var is set
- ❌ Apoptosis wipe — ruled out, #10229 landed protection; nodes exist now
- ❌ Test pollution wipe — ruled out, #10229 testDbPath refactor prevents it
- ❌ Stuck cache (#10181 class) — would manifest as
get_node also failing; it doesn't
- ❌ Missing seed in SQLite —
get_node + manual seedAgentIdentities.mjs run confirm nodes exist with correct createdAt
- ❌ Worktree data isolation — ruled out post-#10224/#10225; main's 48.9 MB SQLite is canonical
Three remaining hypotheses to investigate
Hypothesis A: Silent rejection in GraphService.initAsync's _initPromise
If the self-seed loop or any earlier step in _initPromise's async body throws, the promise rejects. bindAgentIdentity's await GraphService.ready() at Server.mjs:218 sees the rejection, its try/catch at line 227-230 catches it, returns null. The rejection doesn't re-trigger subsequent queries — later get_node calls may re-enter initAsync via some other path, or bypass it since this.db was set before the error.
Specific candidates for silent rejection:
JSON.parse of malformed node data in the self-seed loop's defensive-retention branch (GraphService.mjs:~100) — no try/catch wrapping
storage.db.prepare(...) throwing under concurrent WAL access from Antigravity's hardlinked SQLite process
this.upsertNode post-#10234 triggering getAdjacentNodes → loadNodeVicinitySync which may throw on certain row states
Investigation approach: add temporary logging inside the self-seed loop to log each identity's path (lazy-load triggered, cache-hit/miss, upsert outcome). Boot once. Inspect log order vs. the Identity: ... line from logIdentityStatus.
Hypothesis B: Cross-process hardlinked-SQLite timing race
Main's .neo-ai-data/sqlite/memory-core-graph.sqlite is HARDLINKED with Antigravity's path (verified via stat earlier this session: inode 3091535). Multiple MCP server processes (Claude Code + Antigravity × 2 per harness) write to the same file.
Specific race candidate: Claude Code's MCP initAsync runs its self-seed while Antigravity's SQLite has an open write transaction. Claude Code's self-seed triggers lazy-vicinity → loadNodeVicinitySync → SQLite SELECT that sees the pre-transaction snapshot (empty). bindAgentIdentity's getNode fires immediately after → also sees empty. Later, Antigravity's transaction commits + WAL checkpoints → Claude Code's subsequent queries see full state via new WAL read.
Investigation approach: start Claude Code MCP alone (kill Antigravity processes first). If bind succeeds, race is cross-process. If bind still fails, race is in-process.
Hypothesis C: GraphService.ready() resolves before self-seed completes
Per GraphService.mjs:37-103:
async initAsync() {
await super.initAsync();
if (this.db || this._initPromise) {
if (this._initPromise) await this._initPromise;
return;
}
this._initPromise = (async () => {
})();
await this._initPromise;
}If ready() (inherited from Neo.core.Base) returns without properly awaiting _initPromise in some code path, or if _initPromise resolves in a different microtask boundary than expected, bindAgentIdentity's subsequent getNode could race.
Investigation approach: add logging to mark _initPromise start + _initPromise resolution + bindAgentIdentity entry + bindAgentIdentity getNode completion. Verify strict ordering.
The Architectural Reality
Relevant files + line numbers (at commit 057130b06, dev HEAD as of 2026-04-23 14:02 UTC):
ai/mcp/server/memory-core/Server.mjs:208-231 — bindAgentIdentity implementation
ai/mcp/server/memory-core/Server.mjs:117-123 — boot sequence: resolveStdioIdentity → setStdioIdentityState
ai/mcp/server/memory-core/services/GraphService.mjs:37-103 — initAsync with _initPromise gating
ai/mcp/server/memory-core/services/GraphService.mjs:92-113 — #10232 self-seed loop (after frontier + Neo-Master-Architecture seeds)
ai/graph/identityRoots.mjs — shared IDENTITIES array (4 entries)
ai/graph/Database.mjs:77-112 — syncCache post-#10190 (no longer guards on lastSyncId=0)
ai/graph/Database.mjs:252+ — getAdjacentNodes with lazy-vicinity load
ai/mcp/server/memory-core/services/HealthService.mjs:503-511 — setStdioIdentityState setter (clears cache)
ai/mcp/server/memory-core/services/HealthService.mjs:9-48 — buildIdentityBlock pure projection
Proposed Investigation Steps
- Add temporary boot logging (intentionally noisy, removed before merge):
- Inside #10232 self-seed loop: log each identity's path + SQLite count pre/post upsert
- Inside
bindAgentIdentity: log timing of ready() resolution + getNode result
- Inside
GraphService.initAsync at key boundaries
- Run in isolation: kill Antigravity MCP processes (
pkill -f "antigravity/neomjs/neo/ai/mcp/server/memory-core"), restart Claude Code. If bind succeeds, Hypothesis B confirmed. If still fails, Hypothesis A or C.
- If in-process race: add synchronization checkpoint (e.g., a
await new Promise(resolve => setImmediate(resolve)) between self-seed and initAsync resolve) and retest. If bind succeeds, ordering race. If still fails, Hypothesis A.
- If Hypothesis A: wrap the self-seed loop in try/catch; log any caught error. If error found → fix + retest.
Acceptance Criteria
Out of Scope
- Reinstating the #10182 self-heal block — the "surface, don't obscure" discipline holds; proper fix is at the race, not at the symptom-recovery layer
- SSE transport's per-request identity resolution — different architectural layer
- Changes to the IDENTITIES array itself
- Broader concurrency-model changes to the MCP server (would belong to #10186)
Avoided Traps
- "Just add retry logic to bindAgentIdentity": rejected. That's #10185's pattern which was already removed. Retries mask races rather than fix them; the ADR §5.1.5 rationale + PR #10227 "surface, don't obscure" discipline both apply here. Find the race, don't paper over it.
- "Add a 200ms delay after initAsync": rejected. Arbitrary timing fixes that bug without understanding it will re-appear in different forms later.
- "Revert #10232 self-seed, go back to CLI-only seeding": rejected. Self-seed is architecturally correct (eliminates manual-recovery loop); the race is in the INTERACTION with bindAgentIdentity, not in self-seed itself.
Related
- #10176 (observability — now providing the diagnostic signal): PR #10239 merged
- #10232 (boot-time self-seed): PR #10236 merged — provides the nodes the race misses
- #10234 (upsertNode lazy-load): merged — correct write discipline, but doesn't help if bind runs pre-seed
- #10229 (test pollution + apoptosis exemption): merged — rules out two wipe vectors
- #10227 (#10182 self-heal removal): merged — removed the recovery band-aid; this ticket finds the cause so the band-aid stays gone
- #10186 (MCP concurrency audit epic): parent — this race is within concurrency audit scope
- #10139 (Mailbox A2A primitive): grand-parent — Priority 0 goal closes when this lands
Origin Session ID: 22f662c4-12e5-4360-b207-0280c8aeab41
Handoff Retrieval Hints
Memory Core queries to seed fresh-session context:
query_raw_memories(query="bindAgentIdentity boot-time race healthcheck identity bound false")
query_raw_memories(query="GraphService initAsync self-seed _initPromise timing")
query_raw_memories(query="cross-process hardlinked SQLite Antigravity Claude Code race")
query_summaries(query="A2A operational identity binding boot race post-10239")
Session-ID references for full forensic context:
22f662c4-12e5-4360-b207-0280c8aeab41 — this ticket's session (post-#10239 restart, observed failure)
a7cab439-48cd-4248-bb62-23e6ae2d26e9 — PR #10239 authoring + validation
24aa1fa1-9a22-498e-97f2-760c12e5a79d — wipe-mechanism audit (#10230/#10231/#10232/#10233 filed)
8968b9f6-4606-46e7-a386-7f01b29e2a3a — the original empirical pain that motivated #10232 self-seed
0327771f-b383-472b-8f05-ea8e35e9a65c — original "2-restart recovery loop" narrative
Commit-range anchors:
057130b06 — dev HEAD at observed failure (#10239 merged)
- Prior merges this session: #10234 (
e61667070), #10235 (5631b4872), #10236 (cf1c70489), #10229 (fe513a1bc)
One-liner to reproduce:
"With a fresh MCP boot against a graph containing seeded AgentIdentity nodes, healthcheck.identity.bound is false despite get_node('@neo-opus-ada') returning the node correctly in the same session."
Context
Post-#10239 restart validation (session
22f662c4, 2026-04-23 14:02 UTC) empirically confirmed a boot-time race that the four merged wipe-mechanism defenses (#10229, #10232, #10234, #10235) do not close. The race is upstream of the graph substrate but downstream of the identity resolver. #10176's newhealthcheck.identityblock provides the definitive signal for the first time.This ticket is specifically for a fresh session to deep-dive forensics. All prior diagnostic state is captured below so the next agent doesn't re-derive it.
The Problem
Observed state (session
22f662c4, post-#10239 restart)healthcheck → { "identity": { "source": "env-var", // NEO_AGENT_IDENTITY IS reaching the MCP spawn "bound": false, // bindAgentIdentity returned null "nodeId": null // no binding latched on stdioIdentity } }But at the same session 61 seconds later:
get_node({id: "@neo-opus-ada"}) → { "id": "@neo-opus-ada", "type": "AgentIdentity", "name": "Claude Opus 4.7", "description": "Anthropic Claude Opus version 4.7 Agent Identity" } get_node({id: "AGENT:*"}) → { "id": "AGENT:*", "type": "BroadcastSentinel", ... }The nodes exist with correct types and full seed metadata.
bindAgentIdentitycouldn't find them at boot. The graph query later finds them. This is either a boot-time timing race OR a silent-rejection pattern in the init chain.What does NOT explain it
source: 'env-var'confirms propagationgh-CLItimeout — ruled out, not in resolver chain when env-var is setget_nodealso failing; it doesn'tget_node+ manualseedAgentIdentities.mjsrun confirm nodes exist with correctcreatedAtThree remaining hypotheses to investigate
Hypothesis A: Silent rejection in
GraphService.initAsync's_initPromiseIf the self-seed loop or any earlier step in
_initPromise's async body throws, the promise rejects.bindAgentIdentity'sawait GraphService.ready()atServer.mjs:218sees the rejection, its try/catch at line 227-230 catches it, returns null. The rejection doesn't re-trigger subsequent queries — laterget_nodecalls may re-enterinitAsyncvia some other path, or bypass it sincethis.dbwas set before the error.Specific candidates for silent rejection:
JSON.parseof malformed node data in the self-seed loop's defensive-retention branch (GraphService.mjs:~100) — no try/catch wrappingstorage.db.prepare(...)throwing under concurrent WAL access from Antigravity's hardlinked SQLite processthis.upsertNodepost-#10234 triggeringgetAdjacentNodes → loadNodeVicinitySyncwhich may throw on certain row statesInvestigation approach: add temporary logging inside the self-seed loop to log each identity's path (lazy-load triggered, cache-hit/miss, upsert outcome). Boot once. Inspect log order vs. the
Identity: ...line fromlogIdentityStatus.Hypothesis B: Cross-process hardlinked-SQLite timing race
Main's
.neo-ai-data/sqlite/memory-core-graph.sqliteis HARDLINKED with Antigravity's path (verified viastatearlier this session: inode3091535). Multiple MCP server processes (Claude Code + Antigravity × 2 per harness) write to the same file.Specific race candidate: Claude Code's MCP initAsync runs its self-seed while Antigravity's SQLite has an open write transaction. Claude Code's self-seed triggers lazy-vicinity →
loadNodeVicinitySync→ SQLiteSELECTthat sees the pre-transaction snapshot (empty).bindAgentIdentity'sgetNodefires immediately after → also sees empty. Later, Antigravity's transaction commits + WAL checkpoints → Claude Code's subsequent queries see full state via new WAL read.Investigation approach: start Claude Code MCP alone (kill Antigravity processes first). If bind succeeds, race is cross-process. If bind still fails, race is in-process.
Hypothesis C:
GraphService.ready()resolves before self-seed completesPer
GraphService.mjs:37-103:async initAsync() { await super.initAsync(); if (this.db || this._initPromise) { if (this._initPromise) await this._initPromise; return; } this._initPromise = (async () => { // ... storage setup + frontier seed + #10232 self-seed + FS ingestion })(); await this._initPromise; }If
ready()(inherited fromNeo.core.Base) returns without properly awaiting_initPromisein some code path, or if_initPromiseresolves in a different microtask boundary than expected,bindAgentIdentity's subsequentgetNodecould race.Investigation approach: add logging to mark
_initPromisestart +_initPromiseresolution +bindAgentIdentityentry +bindAgentIdentity getNode completion. Verify strict ordering.The Architectural Reality
Relevant files + line numbers (at commit
057130b06, dev HEAD as of 2026-04-23 14:02 UTC):ai/mcp/server/memory-core/Server.mjs:208-231—bindAgentIdentityimplementationai/mcp/server/memory-core/Server.mjs:117-123— boot sequence:resolveStdioIdentity→setStdioIdentityStateai/mcp/server/memory-core/services/GraphService.mjs:37-103—initAsyncwith_initPromisegatingai/mcp/server/memory-core/services/GraphService.mjs:92-113— #10232 self-seed loop (after frontier + Neo-Master-Architecture seeds)ai/graph/identityRoots.mjs— shared IDENTITIES array (4 entries)ai/graph/Database.mjs:77-112—syncCachepost-#10190 (no longer guards onlastSyncId=0)ai/graph/Database.mjs:252+—getAdjacentNodeswith lazy-vicinity loadai/mcp/server/memory-core/services/HealthService.mjs:503-511—setStdioIdentityStatesetter (clears cache)ai/mcp/server/memory-core/services/HealthService.mjs:9-48—buildIdentityBlockpure projectionProposed Investigation Steps
bindAgentIdentity: log timing ofready()resolution +getNoderesultGraphService.initAsyncat key boundariespkill -f "antigravity/neomjs/neo/ai/mcp/server/memory-core"), restart Claude Code. If bind succeeds, Hypothesis B confirmed. If still fails, Hypothesis A or C.await new Promise(resolve => setImmediate(resolve))between self-seed andinitAsyncresolve) and retest. If bind succeeds, ordering race. If still fails, Hypothesis A.Acceptance Criteria
healthcheck.identity.bound === true→list_messagesreturns without error → first real A2A broadcast to@neo-gemini-prosucceeds (Priority 0 session goal from the 2026-04-22 handoff on #10139 finally closes)logger.debuglevel for future forensicslearn/agentos/tooling/MemoryCoreMcpAuth.md§Troubleshooting extended with the specific failure mode discoveredOut of Scope
Avoided Traps
Related
Origin Session ID:
22f662c4-12e5-4360-b207-0280c8aeab41Handoff Retrieval Hints
Memory Core queries to seed fresh-session context:
Session-ID references for full forensic context:
22f662c4-12e5-4360-b207-0280c8aeab41— this ticket's session (post-#10239 restart, observed failure)a7cab439-48cd-4248-bb62-23e6ae2d26e9— PR #10239 authoring + validation24aa1fa1-9a22-498e-97f2-760c12e5a79d— wipe-mechanism audit (#10230/#10231/#10232/#10233 filed)8968b9f6-4606-46e7-a386-7f01b29e2a3a— the original empirical pain that motivated #10232 self-seed0327771f-b383-472b-8f05-ea8e35e9a65c— original "2-restart recovery loop" narrativeCommit-range anchors:
057130b06— dev HEAD at observed failure (#10239 merged)e61667070), #10235 (5631b4872), #10236 (cf1c70489), #10229 (fe513a1bc)One-liner to reproduce: "With a fresh MCP boot against a graph containing seeded AgentIdentity nodes,
healthcheck.identity.boundisfalsedespiteget_node('@neo-opus-ada')returning the node correctly in the same session."