What is the Neural Link?

The Neural Link is a bi-directional bridge that connects AI agents directly to the Neo.mjs runtime. It lets agents inspect the Scene Graph, component state, event listeners, computed styles, and DOM rectangles, and mutate the running application in real time.

Why is Neo.mjs called an Application Engine instead of a framework?

Neo.mjs maintains persistent application objects in a worker-backed Scene Graph instead of compiling application state away into ephemeral DOM nodes. That architecture enables multi-window orchestration, runtime permutation, and deep AI introspection.

What is Context Engineering?

Context Engineering shapes the information and tool environment around AI agents. Neo.mjs implements it through Knowledge Base, Memory Core, GitHub Workflow, and Neural Link MCP servers for frontier harnesses, plus a File System MCP server for internal Neo.ai.Agent local loops.

What is the Neo.mjs Agent OS?

The Neo.mjs Agent OS is the repository Brain: source code and services for Memory Core, Knowledge Base, Active Hybrid GraphRAG, DreamService, Golden Path synthesis, A2A coordination, and Neural Link tooling.

Neo.mjs - Self-Evolving Software Organism

Frontmatter

title	fix(ai): derive LMS embedding suffix from GGUF metadata (#14015)
author	neo-gpt
state	Merged
createdAt	Jun 25, 2026, 8:36 PM
updatedAt	Jun 25, 2026, 11:03 PM
closedAt	Jun 25, 2026, 11:03 PM
mergedAt	Jun 25, 2026, 11:03 PM
branches	`dev` ← `codex/14015-lms-gguf-sep-metadata`
url	https://github.com/neomjs/neo/pull/14016
contentTrust
projected
quarantined	0
signals	[]

Merged

neo-gpt commented on Jun 25, 2026, 8:36 PM

Resolves #14015

Derives the LM Studio embedding request suffix from the loaded GGUF tokenizer metadata instead of mapping Qwen3 to the chat EOT token. TextEmbeddingService now appends the GGUF EOS token text (<|endoftext|> for the installed Qwen3 embedding GGUF), and the provider-readiness canary can apply the same metadata-derived suffix when LMS metadata is present. Generic OpenAI-compatible and unknown-metadata paths remain unchanged.

Evidence: L3 (live lms ps --json + local GGUF header probe showed eos_token_id -> <|endoftext|> and service-level fake-endpoint request capture sent that token) -> L3 required (runtime request-boundary correctness). Residual: post-merge operator log confirmation that LMS warnings stop during repair-defrag.

Deltas from ticket

The first #14015 body focused on a canary bypass. Live logs then showed the real repair-defrag batch already ended with <|im_end|> and still warned, so the implementation pivoted to the stronger root cause: <|im_end|> is eot_token_id, not this GGUF's EOS/SEP token.

Test Evidence

node --input-type=module -e '...' live helper probe against /Users/tobiasuhlig/.lmstudio/models/Qwen/Qwen3-Embedding-8B-GGUF/Qwen3-Embedding-8B-Q4_K_M.gguf -> suffix <|endoftext|>.
node --input-type=module -e '...' service-path probe with actual lms ps metadata + fake endpoint -> request input hello<|endoftext|> and world<|im_end|><|endoftext|>.
npm run test-unit -- test/playwright/unit/ai/services/memory-core/TextEmbeddingService.retry.spec.mjs test/playwright/unit/ai/scripts/runners/runSandman.spec.mjs -> 87 passed.
git diff --check and git diff --cached --check passed.

Post-Merge Validation

Restart the repair-defrag / embedding sender on merged dev and confirm LM Studio no longer emits the SEP warning for Memory Core re-embed batches.

Authored by Euclid (GPT-5, Codex Desktop). Session 9280140f-8b54-4462-9342-49cca7e226f4.

neo-gpt commented on Jun 25, 2026, 8:54 PM

Fresh operator log evidence confirms the current root cause rather than clearing it: live request bodies can visibly end with <|im_end|> and LM Studio still emits last token ... is not SEP once per embedded string.

Same-turn local falsifier on the loaded Qwen3 embedding GGUF:

tokenizer.ggml.eos_token_id = 151643 -> <|endoftext|>
tokenizer.ggml.eot_token_id = 151645 -> <|im_end|>
tokenizer.ggml.add_eos_token = true

tobiu APPROVED reviewed on Jun 25, 2026, 11:02 PM

No review body provided.