LearnNewsExamplesServices
Frontmatter
id11783
titleSkip no-op KB cascades after primary dev sync
stateClosed
labels
enhancementaiperformancebuild
assigneesneo-gpt
createdAtMay 22, 2026, 10:26 PM
updatedAtMay 22, 2026, 11:09 PM
githubUrlhttps://github.com/neomjs/neo/issues/11783
authorneo-gpt
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtMay 22, 2026, 11:09 PM

Skip no-op KB cascades after primary dev sync

Closed v13.0.0/archive-v13-0-0-chunk-13 enhancementaiperformancebuild
neo-gpt
neo-gpt commented on May 22, 2026, 10:26 PM

Context

The operator surfaced repeated orchestrator child-daemon log output where primary-dev-sync cascades into npm run ai:sync-kb, the KB pipeline rebuilds the full JSONL corpus, fetches every existing Chroma ID, and only then reports:

Knowledge base file created with 25357 chunks.
Fetching existing documents from ChromaDB...
Found 25351 existing documents.
0 chunks to add or update.
0 chunks to delete.
No changes detected. Knowledge base is up to date.

This is real friction: the current zero-change fast path exists, but it fires too late. By the time VectorService.embed() returns No changes detected, the daemon has already paid the source-extraction cost and the Chroma ID-scan cost.

Duplicate sweep:

  • ask_knowledge_base(type='ticket', query='PrimaryRepoSyncService kbSync no changes ai:sync-kb optimization') found adjacent #10088, #8414, #10003/#10572 history, but no focused primary-dev-sync no-op cascade ticket.
  • Local resources/content/issues / resources/content/discussions search found adjacent #11503 / #11520 heavy-maintenance mutex and nested cascade observability work, plus #11135 / #11169 dev-sync-root configuration work. Those are closed or differently scoped.
  • Live GitHub searches for PrimaryRepoSyncService kbSync, ai:sync-kb no changes, primary-dev-sync branch SHA, and orchestrator kbSync no changes found no direct open duplicate. #10088 remains adjacent but broader; it concerns post-merge automation ownership, not avoiding a wasteful already-scheduled no-op cascade.

The Problem

PrimaryRepoSyncService correctly skips when a root is already at origin/dev (behind === 0). But when origin/dev has advanced, a successful fast-forward pull currently calls runKbSync() unconditionally for the owning root.

That is too coarse. A branch SHA change does not necessarily mean the KB corpus changed. The pulled commits might touch files outside the KB source registry, or might update generated/non-indexed state. In that case the cascade still shells out to npm run ai:sync-kb, which runs DatabaseService.syncDatabase():

  1. createKnowledgeBase() extracts all registered sources and writes dist/ai-knowledge-base.jsonl.
  2. VectorService.embed() reads that JSONL, fetches all existing Chroma IDs, computes the diff, and only then returns No changes detected.

The result is a repeated daemon-pressure cycle: work that is technically correct but avoidable.

The Architectural Reality

Relevant current code:

  • ai/daemons/services/PrimaryRepoSyncService.mjs
    • syncDevRoot() fetches origin/dev, checks behind, fast-forwards, and calls runKbSync() after a successful pull.
    • syncConfiguredDevRoots() calls the cascade once from the owner root if at least one configured root completed.
    • runKbSync() shells out to npm run ai:sync-kb and annotates the nested kbSync lifecycle per #11520.
  • buildScripts/ai/syncKnowledgeBase.mjs
    • Wraps the CLI with withHeavyMaintenanceLease() and calls KB_DatabaseService.syncDatabase().
  • ai/services/knowledge-base/DatabaseService.mjs
    • syncDatabase() always runs createKnowledgeBase() before embedKnowledgeBase().
  • ai/services/knowledge-base/VectorService.mjs
    • embed() has the late zero-change path after it has loaded generated JSONL and fetched existing Chroma IDs.

Important adjacent tickets:

  • #10088: broader post-merge KB automation and single-owner semantics; do not reintroduce per-harness boot sync.
  • #11503 / #11520: heavy-maintenance mutex and nested cascade observability; do not regress lifecycle annotation or lease behavior.
  • #11135 / #11169: multi-root dev-sync configuration; the cascade remains owner-root based and single-run per primary-dev-sync cycle.

The Fix

Add an early no-op gate for primary-dev-sync KB cascades.

Recommended shape:

  1. Capture oldHead before the fast-forward pull and newHead after the pull.
  2. Persist the last successfully evaluated KB-cascade state for the owning checkout, including at minimum headSha, checkedAt, and whether a KB cascade actually ran or was skipped as no-KB-impact.
  3. Before spawning npm run ai:sync-kb, evaluate whether oldHead..newHead contains any KB-relevant input changes.
  4. If no KB-relevant source inputs changed, skip runKbSync() and record an operator-visible kbSync: false, reasonCode: 'no-kb-relevant-changes', oldHead, newHead, and changed-file count/details bounded for logs.
  5. If the relevance check is unavailable, ambiguous, or sees KB-relevant changes, fall back to the current safe behavior and run the cascade.

The lowest-risk implementation is likely in PrimaryRepoSyncService, not VectorService: the service already owns oldHead/newHead, pull outcome, health details, and the decision to call runKbSync(). VectorService's late no-change diff should remain as a correctness backstop.

The KB-relevance predicate must be conservative. Possible implementation options to evaluate during intake:

Option Shape Pros Risk / fallback
A — Path allowlist from current source families Match git diff --name-only oldHead..newHead against known KB source roots (src, learn, .agents/skills, resources/content, tests, release notes, API docs, etc.) Cheap and easy to unit-test Must stay aligned with SourceRegistry; ambiguous paths should run sync
B — SourceRegistry exposes input globs / roots Each KB source declares its watched path predicates Most correct long-term contract Wider refactor; probably not needed for first pass
C — Corpus fingerprint cache Compute a cheap source-input fingerprint before full JSONL creation More general than path allowlist Can drift into a second KB compiler; avoid if it duplicates extraction logic

Lean: start with A, keep B as a future refinement if SourceRegistry/custom-source support makes the allowlist brittle.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
PrimaryRepoSyncService KB cascade decision This ticket + #11135/#11520 behavior Run ai:sync-kb only after a successful pull with KB-relevant input deltas, or when relevance is unknown Existing unconditional cascade Service JSDoc + PR body Unit tests using execFileSyncFn seam
Health / task outcome details Existing HealthService.recordTaskOutcome() shape Surface skipped cascade as success/skip metadata, not failure Sparse log only Operator log examples if docs touched Unit assertions on details payload
Nested kbSync lifecycle annotation #11520 Preserve kbSync child lifecycle events when cascade runs No event when deliberately skipped, with parent details carrying skip reason JSDoc Regression tests for run and skip paths
KB diff correctness VectorService.embed() late no-change path Remains correctness backstop If early gate misclassifies unknown as relevant, run full sync None required Existing VectorService tests plus focused service tests

Acceptance Criteria

  • AC1 — PrimaryRepoSyncService records old/new dev HEADs around a successful fast-forward pull.
  • AC2 — The service skips runKbSync() when the pull contains no KB-relevant input changes, and records an explicit reasonCode: 'no-kb-relevant-changes' (or equivalent stable code) in task/health details.
  • AC3 — The skip path does not mark the parent primary-dev-sync as failed and does not emit a misleading nested kbSync running/completed lifecycle event.
  • AC4 — The run path still preserves #11520 nested kbSync lifecycle annotation when relevant files changed.
  • AC5 — The predicate is conservative: unknown/unsupported diff states, missing old head, first run, or predicate errors fall back to running ai:sync-kb.
  • AC6 — Configured multi-root mode keeps the single owner-root cascade semantics from #11135: one KB decision per primary-dev-sync cycle, not one per clone.
  • AC7 — Unit tests cover: no behind skip; behind with no KB-relevant files; behind with KB-relevant files; metadata-only resolution path; configured roots with one completed root but no KB-relevant changes; fallback-on-unknown.
  • AC8 — The existing late VectorService.embed() zero-change behavior remains intact as a backstop; this ticket only avoids invoking it unnecessarily from primary-dev-sync.

Out of Scope

  • Reworking post-merge KB automation in #10088.
  • Reintroducing per-harness boot-time KB sync.
  • Changing embedding provider, vector dimensions, Chroma collection strategy, or stale-delete semantics.
  • Removing the heavy-maintenance lease from syncKnowledgeBase.mjs.
  • Running an independent KB sync for every configured clone.
  • A broad SourceRegistry refactor unless intake proves the path predicate cannot be kept conservative.

Avoided Traps

  • Branch-SHA-only skip: rejected as too weak. It prevents repeated work for the exact same head, but still runs full sync for every non-KB commit.
  • Replace VectorService diffing: rejected. The late zero-change path is still the correctness backstop.
  • Assume all dev commits affect KB: rejected by the observed log; the daemon already found zero changed chunks after a full pass.
  • Per-root cascade in multi-root mode: rejected by #11135; the KB cascade is owner-root based and should remain single-run.

Related

  • Adjacent: #10088 — post-merge KB sync trigger automation.
  • Adjacent/closed: #11503 — heavy-maintenance mutex.
  • Adjacent/closed: #11520 — nested PrimaryRepoSyncService.runKbSync() lifecycle annotation.
  • Adjacent/closed: #11135 — configured dev auto-sync roots.
  • Adjacent/closed: #11169 — local config for orchestrator dev-sync roots.

Origin Session ID: d60db68f-8ff0-48a6-b168-237ca9dca2a0

Handoff Retrieval Hint: query_raw_memories(query="PrimaryRepoSyncService kbSync no changes oldHead newHead no-kb-relevant-changes")

tobiu referenced in commit fbaf0a0 - "fix(ai): skip no-op primary-dev KB cascades (#11783) (#11784) on May 22, 2026, 11:09 PM
tobiu closed this issue on May 22, 2026, 11:09 PM