LearnNewsExamplesServices
Frontmatter
id12244
titleAdd identity fact coherence checks
stateClosed
labels
documentationenhancementaitestingbuild
assigneesneo-gpt
createdAtMay 31, 2026, 7:29 AM
updatedAtMay 31, 2026, 4:53 PM
githubUrlhttps://github.com/neomjs/neo/issues/12244
authorneo-gpt
commentsCount1
parentIssue12225
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[x] 12231 Fix drifted identity facts across machine-facing surfaces
blocking[]
closedAtMay 31, 2026, 4:53 PM

Add identity fact coherence checks

Closed v13.0.0/archive-v13-0-0-chunk-15 documentationenhancementaitestingbuild
neo-gpt
neo-gpt commented on May 31, 2026, 7:29 AM

Parent epic: #12225

Context

#12231 converges the current drifted identity facts across machine-facing and docs surfaces, but it also confirms that some mirrors are still protected by manual review rather than a durable guard. ADR 0018 says facts must converge to a single source of truth; manual convergence is useful once, but future Node/MCP count drift needs a repeatable check.

Creation V-B-A:

  • package.json now exposes the Node floor via engines.node and the functional MCP manifest via scripts matching ai:mcp-server-*.
  • #12231 updates release/generator seams where practical (buildScripts/release/prepare.mjs, buildScripts/docs/seo/generate.mjs), but prose/config-template mirrors can drift again without CI feedback.
  • Duplicate sweep: Knowledge Base MCP was unavailable (Failed to access collections); exact rg over local issue/discussion archives found no dedicated fact-coherence ticket; live gh issue list surfaced #12231 and closed #12203 only, not a follow-up for Node/MCP coherence checks.

The Problem

Identity FACTS are single-valued. The current run fixed stale values, but reviewers still have to remember to grep for old Node floors, stale MCP-server counts, and template omissions. That is exactly the drift class ADR 0018 tries to move out of human memory and into derivation or coherence tooling.

Without a check, a future MCP server addition, Node floor bump, or generated-surface reshape can silently reintroduce the same mismatch across package.json, config templates, docs prose, portal text, and generated SEO output.

The Architectural Reality

ADR 0018 separates FACTS from FRAMING. This ticket is FACTS-only. It should not rewrite the canonical apex, package keywords, or prose positioning.

Current source-of-truth candidates:

  • Node requirement: package.json engines.node.
  • Functional MCP server set: package.json scripts named ai:mcp-server-*, with the exclusion rule captured by the script surface rather than naive ls ai/mcp/server/.
  • Generated SEO output: source is buildScripts/docs/seo/generate.mjs, not apps/portal/llms.txt directly.
  • Release-time version mirrors: source is package.json plus buildScripts/release/prepare.mjs.

The Fix

Add a focused identity-facts coherence check that can run in CI or existing lint pathways. The check should derive the current Node floor and functional MCP server set from their source surfaces, then scan known mirrors for stale or missing values.

Expected scope:

  1. Implement a small checker in the existing build/lint script substrate, following sibling patterns.
  2. Check package.json / package-lock.json engine coherence.
  3. Check .codex/config.template.toml and .gemini/settings.template.json include the same functional MCP server set as package.json scripts.
  4. Check current known prose/generated-source mirrors for stale exact phrases, but avoid brittle broad prose parsing where source text can legitimately be audience-segmented.
  5. Wire the check into the appropriate existing lint path only if the signal is high and false positives are low; otherwise expose it as an operator-run script and document why.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
Node floor coherence package.json engines.node package-lock.json root package and known docs mirrors agree with the SSOT Emit actionable failure naming stale files PR body / script help text Unit or script run with pass/fail fixtures
Functional MCP server set package.json ai:mcp-server-* scripts Config templates and explicit count prose agree with the derived set/count Allow explicit ignore list with rationale for non-functional/PoC surfaces neo-identity-update facts ledger if behavior changes Script output showing expected vs actual
Generated SEO fact mirrors buildScripts/docs/seo/generate.mjs Check generator text, not generated output as source If generated output is checked, message must point back to generator PR body Regenerate or check command evidence

Decision Record Impact

Aligned-with ADR 0018. No ADR change expected unless implementation discovers a different SSOT than package.json scripts for the functional MCP set.

Acceptance Criteria

  • A repeatable command detects Node-floor drift between package.json, package-lock.json, and known mirrors.
  • A repeatable command detects functional MCP-server set/count drift between package.json scripts and MCP config templates.
  • The check's failure output names the stale surface and the source-of-truth value.
  • The check avoids naive ls ai/mcp/server/ counting and preserves the shared/ / gitlab-workflow exclusion rule.
  • The implementation either wires into the repo lint path with low false-positive risk or documents why it remains an explicit operator-run script.
  • Cross-family review completed before merge because this touches identity-maintenance substrate.

Out of Scope

  • Package keyword ordering/inclusion; tracked separately by #12241.
  • Changing the canonical apex or any framing cluster.
  • Adding new MCP servers.
  • Rewriting all identity prose to a single phrase.

Avoided Traps

  • Treating framing like facts.
  • Counting directories under ai/mcp/server/ and accidentally including infra/PoC folders.
  • Checking generated llms.txt as the source while ignoring generate.mjs.
  • Adding a noisy CI gate that blocks legitimate audience-segmented prose.

Related

  • Parent: #12225
  • Follow-up from: #12231
  • Related: #12241 (package keyword priority audit)
  • ADR 0018: Neo Identity Source-of-Truth Model

Origin Session ID: 87203e0b-c9ff-4a82-a67b-9a0dda760c32 Retrieval Hint: "identity facts coherence check Node engines MCP server count package scripts config templates ADR 0018"

tobiu referenced in commit 4f8cb23 - "test(identity): add fact coherence check (#12244) (#12260) on May 31, 2026, 4:53 PM
tobiu closed this issue on May 31, 2026, 4:53 PM