Orchestrator should manage Memory Core Chroma daemon (port 8001)

Context

npm run ai:orchestrator is the operator-facing entry point for the Agent OS maintenance daemon. The orchestrator's TaskDefinitions.mjs (ai/daemons/TaskDefinitions.mjs) currently manages:

• chroma → KB Chroma on port 8000, path .neo-ai-data/chroma/knowledge-base
• bridgeDaemon → bridge daemon
• mlx → mlx_lm inference (currently failing — separate ticket)
• summary → session summarization (periodic)
• kbSync → knowledge base sync (periodic)
• backup → memory core backup (periodic)
• primary-dev-sync, dream, golden-path (service tasks)

The Memory Core MCP server's Chroma instance on port 8001 (path .neo-ai-data/chroma/memory-core) is NOT managed by the orchestrator. The MC server's config.mjs defaults autoStartDatabase: false, meaning no process auto-starts MC's Chroma.

Result: operators who run npm run ai:orchestrator for the first time get a healthy KB substrate but the MC server returns unhealthy (Chroma not accessible) at every healthcheck until they manually run ./node_modules/.bin/chroma run --path .neo-ai-data/chroma/memory-core --port 8001 in a separate terminal.

Empirical anchor (2026-05-14T23:24Z, operator's first ever npm run ai:orchestrator run):

• Orchestrator booted successfully (PID 2278)
• KB Chroma child PID 2279 alive (lsof -nP -iTCP:8000)
• MC healthcheck: status: unhealthy, details: ["Database engine not accessible: ChromaDB is not accessible"]
• lsof -nP -iTCP:8001: no listener
• Manual chroma run --path .neo-ai-data/chroma/memory-core --port 8001 restored MC health within 5s

Problem

The orchestrator's job is to be the single-writer process supervisor for the Agent OS substrate. Managing 1 of 2 Chroma instances violates that invariant — the other instance is operator-burden, which:

• Causes silent boot failure (operator doesn't know MC needs a separate process until they hit the unhealthy healthcheck)
• Breaks first-run UX (npm run ai:orchestrator should "just work" for the full substrate, not require a magic side-task)
• Inverts the single-writer principle (orchestrator is supposed to OWN process lifecycle for shared substrate)

Architectural Reality

Two fix shapes considered:

• Option A (preferred): Add mcChroma to TaskDefinitions.mjs alongside the existing chroma task. Orchestrator becomes the single-writer for both Chroma instances. Orchestrator.poll()'s continuousTasks array (['chroma', 'bridgeDaemon', 'mlx']) extends to ['chroma', 'mcChroma', 'bridgeDaemon', 'mlx'].

• Option B (rejected): Flip autoStartDatabase: true in MC config.mjs. Makes the MC server self-bootstrap. Conflicts with multi-instance deployment topology (isPrimary flag in MC config) — multiple MC server processes spawning competing Chroma instances is exactly the race the orchestrator exists to prevent.

Option A preserves the orchestrator's role as single-writer; Option B re-introduces the race condition #9942 originally fixed.

Fix (concrete prescription)

In ai/daemons/TaskDefinitions.mjs buildTaskDefinitions() add a mcChroma task entry next to the existing chroma entry:

mcChroma: {
    label          : 'memory core chroma daemon',
    command        : path.resolve(scriptDir, '../../node_modules/.bin/chroma'),
    args           : ['run', '--path', '.neo-ai-data/chroma/memory-core', '--port', '8001'],
    pidFileName    : 'mc-chroma.pid',
    expectedCommand: 'chroma'
},

In ai/daemons/Orchestrator.mjs#poll extend the continuous-tasks supervisor loop:

const continuousTasks = ['chroma', 'mcChroma', 'bridgeDaemon', 'mlx'];

Acceptance Criteria

• TaskDefinitions.mjs declares mcChroma with the prescribed path + port + pidFileName.
• Orchestrator.poll() continuousTasks includes 'mcChroma'.
• After npm run ai:orchestrator, lsof -nP -iTCP:8001 -sTCP:LISTEN returns a Chroma listener within 10s.
• MC server healthcheck returns status: healthy without any operator-side manual chroma launch.
• Existing chroma task on port 8000 continues to run (KB substrate unaffected).
• PID-file isolation: mc-chroma.pid distinct from chroma.pid so the two tracking files don't collide.
• Recovery on orchestrator restart: if mc-chroma.pid references a live external Chroma process, the orchestrator adopts it (per existing recoverTask semantics).

Out of Scope

• mlx inference task fixing — separate ticket (model-arg format issue).
• ProcessSupervisorService stdio: 'ignore' child-stderr-swallow — separate ticket (debugging substrate).
• Embedding provider (LM Studio :1234) auto-start — separate decision; operator currently runs it via a desktop app, may stay outside orchestrator scope.

Avoided Traps

• Option B (autoStartDatabase: true): rejected. Multi-MC-instance deployments would race for Chroma on port 8001.
• Magic-port hardcoding: prefer reading port + path from MC config.mjs rather than hardcoding 8001 / .neo-ai-data/chroma/memory-core in TaskDefinitions.mjs — but that requires SDK-config-cross-import which may not exist yet. Pragmatic baseline: hardcode matching MC's current defaults; file follow-up if config-import path lands per #10103.
• Adding mcChroma without PID-file isolation: would conflict with chroma.pid and either silently overwrite or fail singleton-enforcement.

Related

• Substrate authority: ai/daemons/Orchestrator.mjs, ai/daemons/TaskDefinitions.mjs, ai/daemons/services/ProcessSupervisorService.mjs
• MC config: ai/mcp/server/memory-core/config.mjs (defaults autoStartDatabase: false; defines engines.chroma.{host, port, dataDir})
• #9942 — original daemon-collision fix that the orchestrator-centralization here preserves
• #11009 — orchestrator class refactor that established the Neo daemon shape

Origin Session

• Origin Session ID: e095c569-beac-4743-998f-e07d4344492e

Retrieval Hint

Search for orchestrator MC Chroma port 8001 autoStartDatabase single-writer mcChroma TaskDefinitions.