Cover full KB sync in oversized-source splitting

Context

Post-merge validation for #14000 / PR #14003 failed during the live Agent OS restart on 2026-06-25. The operator observed the orchestrator Knowledge Base sync still emitting the pre-provider skip for the same oversized source that #14000 was supposed to recover:

source: 'resources/content/issues/chunk-2/issue-12065.md',
inputBytes: 91892,
inputTokensEstimate: 30631,
safeProcessingLimitTokens: 28672,
contextLimitTokens: 32768
[WARN] [VectorService] Skipping over-budget embedding chunk before provider invocation.

PR #14003 is merged and correctly closed #14000, but its body left this post-merge validation item unchecked: run a KB sync including resources/content/issues/chunk-2/issue-12065.md and verify rows exist with oversizedSplit metadata instead of a skip. The live restart supplied that validation, and it failed.

Release classification: boardless Agent OS stabilization follow-up. This is not a v13 release-board attachment; it is a regression follow-up to a closed bug and active #13999 stabilization work.

Duplicate sweep:

• Live latest-open sweep: checked the latest 30 open GitHub issues at 2026-06-25T13:48:50Z. No open issue covers the PR #14003 route-coverage gap. Closest active issue remains #13999, which owns Memory Core backup exportability, not this KB sync route miss.
• Exact GitHub search for oversized Knowledge Base VectorService syncDatabase issue-12065 returned no all-state duplicate beyond the closed #14000 context.
• A2A in-flight sweep: checked latest 30 all-status messages immediately before filing; no competing [lane-claim] or [lane-intent] for this route-coverage follow-up was present.
• Semantic prior-art sweep: Knowledge Base search surfaced #14000 and older KB route/openapi tickets, but no open successor for the failed full-sync validation.

The Problem

#14000 fixed one ingestion path, not the path that the orchestrator uses for full Knowledge Base sync.

The merged code splits oversized chunks inside KnowledgeBaseIngestionService.filterEmbeddingInputBudget() before that service writes temp JSONL and calls VectorService.embed(). The orchestrator kbSync task does not go through that service. It runs the maintenance sync path, which rebuilds aiConfig.dataPath and then calls DatabaseService.syncDatabase().

That means the full-corpus sync still reaches VectorService.embed() with oversized source chunks produced by the source extractors, and VectorService.embedChunks() still performs the final guardrail skip. The guardrail is correct as a last refusal point, but it should not be the first and only protection for recoverable full-sync content.

The result is exactly what the operator observed: resources/content/issues/chunk-2/issue-12065.md is still skipped instead of split during live full KB sync, so the Knowledge Base remains incomplete for that source even after #14003 merged.

The Architectural Reality

Verified current surfaces:

• ai/daemons/orchestrator/taskDefinitions.mjs defines the kbSync task as ai/scripts/maintenance/syncKnowledgeBase.mjs.
• ai/services/knowledge-base/DatabaseService.mjs runs syncDatabase() as createKnowledgeBase() followed by embedKnowledgeBase().
• DatabaseService.embedKnowledgeBase() delegates directly to VectorService.embed(aiConfig.dataPath, {viaMcp, staleStrategy}).
• ai/services/knowledge-base/KnowledgeBaseIngestionService.mjs owns the #14003 split implementation in filterEmbeddingInputBudget() / splitOversizedEmbeddingChunk().
• ai/services/knowledge-base/VectorService.mjs still owns the final pre-provider refusal log: [VectorService] Skipping over-budget embedding chunk before provider invocation.

So the current architecture has two write routes into KB embedding:

1. Tenant/raw ingestion route: KnowledgeBaseIngestionService.ingestSourceFiles() -> split-safe after #14003.
2. Full-corpus sync route: DatabaseService.syncDatabase() -> VectorService.embed() -> still skip-only for oversized full-sync chunks.

The Fix

Close the route-coverage gap without weakening the guardrail.

Expected shape:

1. Make the full-corpus sync route apply the same deterministic oversized-source splitting before VectorService.embedChunks() reaches provider invocation.
2. Keep VectorService as the final refusal guardrail for genuinely unsplittable or still-over-budget chunks.
3. Prefer a shared helper or service-owned normalization seam over copy/pasting the #14003 splitter logic into multiple places.
4. Preserve deterministic IDs and trace metadata (oversizedSplit*) so unchanged oversized sources do not churn Chroma rows.
5. Add a focused test that exercises the full-sync or VectorService.embed() route with an issue-12065.md-class chunk. A test that only calls KnowledgeBaseIngestionService.ingestSourceFiles() is insufficient; that is the blind spot #14003 left behind.
6. Add a post-merge validation command or script path that proves the live sync no longer logs the skip for resources/content/issues/chunk-2/issue-12065.md and produces KB rows for that source.

Implementation note: if a new .mjs helper is introduced, run structural-pre-flight first. A narrow in-place extraction under the existing KB service/helper ownership boundary is likely enough.

Contract Ledger Matrix

Decision Record impact

Aligned with ADR 0019. The fix should read resolved AiConfig leaves from the owning config surface and must not raise context limits, invent hidden defaults, mutate runtime config, or bypass the Provider SSOT to hide the oversized input.

Acceptance Criteria

• The full-corpus KB sync route no longer skips recoverable oversized chunks such as resources/content/issues/chunk-2/issue-12065.md before attempting deterministic splitting.
• The existing final VectorService pre-provider guardrail remains active and refuses genuinely unsplittable or still-over-budget inputs.
• The full-sync route preserves deterministic chunk identity and trace metadata for generated sub-chunks.
• Focused tests cover the route that #14003 missed: DatabaseService.syncDatabase() / embedKnowledgeBase() / VectorService.embed(), not only KnowledgeBaseIngestionService.ingestSourceFiles().
• Test evidence includes an issue-12065.md-class oversized markdown fixture or the real source under controlled limits.
• PR evidence documents the live validation command or post-merge smoke needed to prove the orchestrator no longer logs the issue-12065.md skip.
• No embedding provider limits are raised and no oversized input is sent to the provider as a workaround.

Out of Scope

• Reopening #14000.
• Solving the Memory Core backup exportability corruption tracked by #13999.
• Raising safeProcessingLimitTokens / contextLimitTokens to mask the route bug.
• Removing VectorService's final skip guardrail.
• Rebuilding all KB source parsers unless a specific parser proves necessary for the full-sync path.

Avoided Traps

• Do not accept a unit test that only proves KnowledgeBaseIngestionService; that is already the merged #14003 path.
• Do not treat the live skip as stale checkout evidence. The current failure happened after the merged PR and at the same source path named in #14003's post-merge validation.
• Do not collapse this back into #13999. The backup issue is Memory Core exportability; this follow-up is KB corpus completeness on the full-sync route.

Related

Related: #13999 Related: #14000 Related: #13930 Related: #13929 Related: PR #14003

Origin Session ID: 9280140f-8b54-4462-9342-49cca7e226f4

Handoff Retrieval Hints

• query_raw_memories("issue-12065 PR 14003 VectorService syncDatabase route coverage")
• query_raw_memories("KnowledgeBaseIngestionService splitOversizedEmbeddingChunk full KB sync still skips")
• Exact anchors: DatabaseService.syncDatabase, DatabaseService.embedKnowledgeBase, KnowledgeBaseIngestionService.filterEmbeddingInputBudget, VectorService.embedChunks, resources/content/issues/chunk-2/issue-12065.md