LearnNewsExamplesServices
Frontmatter
id7474
titleImprove AI Query Scoring to Boost Content Matches
stateClosed
labels
enhancementhelp wantedgood first issuehacktoberfestai
assigneestobiu
createdAtOct 13, 2025, 11:20 AM
updatedAtNov 2, 2025, 10:29 AM
githubUrlhttps://github.com/neomjs/neo/issues/7474
authortobiu
commentsCount1
parentIssue7296
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtNov 2, 2025, 10:29 AM

Improve AI Query Scoring to Boost Content Matches

Closed v11.0.0 enhancementhelp wantedgood first issuehacktoberfestai
tobiu
tobiu commented on Oct 13, 2025, 11:20 AM

The AI query system (queryKnowledgeBase.mjs) currently uses a hybrid approach of semantic vector search followed by a keyword-based scoring boost. However, the keyword boosting logic has a significant flaw: it only checks for keywords in a chunk's metadata (e.g., filename, title, path) and completely ignores the chunk's actual content.

This leads to counter-intuitive results. For example, a query for the word "Tron" correctly identifies the blog post ai-native-platform-answers-questions.md (which contains the word in its content) through semantic search. However, it ranks it lower than other articles that are semantically related but do not contain the word at all. This is because those other articles get higher initial semantic scores, and the target article gets no keyword boost to compensate, as the current logic doesn't reward it for having a literal match in its content.

This ticket is to fix the scoring algorithm to give a significant boost to any chunk where the search term is found directly in the content.

Acceptance Criteria

  1. The queryKnowledgeBase.mjs script must be modified.
  2. Inside the keyword scoring loop (queryWords.forEach), a new condition must be added.
  3. This condition will check if the keywordSingular exists within the chunk's content (e.g., metadata.content.toLowerCase().includes(keywordSingular)).
  4. If a match is found, a substantial score boost (e.g., score += 100) should be applied. This boost should be high enough to ensure a literal content match is prioritized over a purely semantic one.
  5. After the change is implemented, running the command npm run ai:query -- -q "Tron" -t blog must rank /Users/Shared/github/neomjs/neo/learn/blog/ai-native-platform-answers-questions.md as the top result.
tobiu assigned to @tobiu on Oct 13, 2025, 11:20 AM
tobiu added parent issue #7296 on Oct 13, 2025, 11:20 AM
tobiu added the enhancement label on Oct 13, 2025, 11:20 AM
tobiu added the help wanted label on Oct 13, 2025, 11:20 AM
tobiu added the good first issue label on Oct 13, 2025, 11:20 AM
tobiu added the hacktoberfest label on Oct 13, 2025, 11:20 AM
tobiu added the ai label on Oct 13, 2025, 11:20 AM
tobiu
tobiu Oct 13, 2025, 11:22 AM

I am reserving this ticket for @SarthakBorude for 7 days.

tobiu cross-referenced by #7467 on Oct 13, 2025, 11:34 AM
tobiu closed this issue on Nov 2, 2025, 10:29 AM