Resolves #16717

Refs #16690 Related: #16706 Related: #16692

#16690 is deliberately NOT the close target. #16690's live ACs still require embeddings to become queryable without a further ingest run, plus pending-drain depth and oldest-age observability. This PR delivers the deferral half only, split out as #16717; the async-drain and drain-observability half remains open on #16690. Flagged by @neo-gpt in review, and he is right — my scope-reduction prose argued the WAL away but never amended those ACs, so closing on this head would have silently retired work nobody decided to drop. Splitting the delivered leaf is the workflow's own remedy for that, and it also resolves the tension the Refs-only fix created: an agent PR requires a standalone Resolves, so the honest answer was a close target that did not exist yet, not a weaker keyword.

An ingest run had two outcomes and needed three. Any error in the summary failed the whole run, so one slow embedding discarded the checkpoint for every chunk that did embed, the repo took a backoff step, and the corpus never grew. classifyIngestionOutcome adds deferred — incomplete, not failed: the checkpoint holds, consecutiveFailures is neither incremented nor reset, and lastRunAttemptAt advances. The deferral does not return the lane to base cadence — the retained streak still governs, and the merged #16692 recovery generation is the single resumption authority that may bypass it once. A deferrable embedding cause now arms that same episode, so deferral hands the recovery lane a reason instead of inventing a second scheduler.

Evidence: L2 (pure classifiers plus the real runTask sweep and the durable per-repo manifest, all exercisable in-process) → L2 required (every criterion this PR claims is an in-process scheduling decision or a durable record). Residual: #16690's async-drain and drain-observability ACs are explicitly NOT delivered here and stay open on the ticket.

Deltas from ticket

Two, both recorded on the ticket body before this PR.

• No write-ahead store. The ticket prescribed one, mirroring memoryWalStore, plus config leaves and a drain daemon. It is unnecessary: VectorService.mjs:1103-1106 only pushes a chunk into chunksToProcess when !existingIds.has(chunkId), with existingIds read from the corpus-scoped collection and chunk ids content-derived. Already-embedded chunks are never re-embedded, so ingest is incremental by construction and a later run resumes rather than restarts — the durable partial progress the ticket wanted already lives in the vector store. The ticket's AC1 ("parsed chunks durably recorded") was over-specified: chunks are derivable from the repo at the checkpoint revision. (The behaviour is asserted in a docblock at VectorService.mjs:146; I traced the branch rather than citing the docblock, having had three ticket premises falsified this session for the reverse.)
• Deferral is opt-in by domain, not merely default. The ticket framed the discriminator as "rejected fails, everything else defers". Applied to a real ingestion summary that is a defect: the stream is mixed — 14 distinct errors.push sites in IngestionService, only two of them the embed path. A parse failure routed through a deferral-by-default classifier would defer forever, never failing, never advancing, never surfacing a cause. Silently stuck is worse than loudly broken, so isEmbedFailureCode gates the domain and the disposition decides within it.

Architectural notes

Why deferral is the default within the embed domain. The obvious design defers only on recognised-transient codes. It would not have fired on the failure that motivated this: the deployment reported KB_VECTOR_EMBED_FAILED, which is the unclassified sentinel — the provider's code matched no entry in either vocabulary. A transient-allow-list rejects precisely the specimen it was built to survive, and silently, because an unrecognised code looks like a decision rather than a gap.

The every is the safety argument. A run defers only when every error is a deferrable embed failure. One rejected code, one non-embed error, or one error carrying no code at all (isEmbedFailureCode(undefined) is false) drops to the failure path unchanged. A codeless error is unclassifiable, and unclassifiable must fail loudly rather than wait.

Deferrals are counted. Without deferredCount, a sweep deferring every repo reports 1 completed, 0 failed and reads as a clean cycle — the same empty-is-not-success defect this ticket family exists to close, landing on the one line an operator scans.

The domain set is derived, not restated. Built from the module's own three sources, so adding a provider mapping widens it in the same edit. A hand-maintained duplicate would have been one rename from silently refusing deferral to a code that classifies fine.

Test Evidence

• ai/services/knowledge-base/helpers/embedFailureClassification.mjs + ai/daemons/orchestrator/services/TenantRepoSyncService.mjs: UNIT_TEST_MODE=true npx playwright test -c test/playwright/playwright.config.unit.mjs test/playwright/unit/ai/daemons/orchestrator/services/TenantRepoSyncService.spec.mjs test/playwright/unit/ai/services/knowledge-base/embedFailureClassification.spec.mjs — 141 passed at the composed head.
• Mutation-proved, both commits, reverted after each with a residue grep:
  • transient-allow-list disposition → 4 red (unclassified specimen, translate-then-dispose composition, non-vacuity control, totality).
  • unconditional deferral → 4 red (three rejected codes + the non-vacuity control, which fires against both mutations, as a control should).
  • domain check removed → the mixed-stream spec reds.
  • deferrable = false (the old two-outcome behaviour) → the new deferred-outcome spec reds.
• Coverage moved rather than deleted: two specs drove the failure path using the now-deferrable KB_VECTOR_EMBED_FAILED. They now use a rejected code, so "an error-bearing summary fails and earns a backoff step" stays pinned for the case it still governs. The #16647 credential-boundary specs were checked and are unaffected — the deferred branch still carries lastSourceErrorCode into the repo state they assert on.

Post-Merge Validation

• On the next external-plane sync sweep, confirm a repo whose embedding is starved reports status: deferred with its consecutiveFailures unchanged, rather than climbing toward the cadence cap.
• Confirm the cycle-summary line renders the N deferred segment on a real sweep.
• This makes a starved provider survivable, not fast. The corpus only grows once the deployment's embedding capacity is addressed — that is a deployment-side item on #16706 and no ticket here closes it.

Commits

• isEmbedFailureCode: deferral opt-in by embed domain, because the summary stream is mixed.
• classifyIngestionOutcome: the third outcome, the caller's deferred persistence branch, and deferredCount.
• Review composition (@neo-gpt RAs 1–3 + design decision (a)): deferral arms the shared recovery episode via the generalized buildEmbeddingRecoveryEpisode; an all-deferred sweep reports deferred rather than completed; classifyEmbeddingRecoveryState inspects recovery BEFORE the failure count, so a repo that deferred at streak 0 still classifies; the deferred projection publishes recoveryState; close target corrected to Refs.

Review composition (@neo-gpt)

Three RAs, all correct, all confirmed in source before fixing:

1. Retained streak vs cadence. The deferred branch preserves consecutiveFailures, and isRepoDue multiplies by 2^13 for the production specimen — so my "returns at base cadence" claim was false. My cohort could not catch it: every repo started at streak 0, where the multiplier is 1 and the capped case does not exist. Fixed by composing with #16692 rather than adding a cadence exception — he explicitly corrected his own RA-1 wording after #16712 merged, and the merged canary → durable-generation → isRepoDue bypass is now the sole resumption authority.
2. All-deferred aggregate status. attemptedCount = completed + failed excluded deferrals, so an all-deferred sweep took the attemptedCount === 0 branch meant for "every repo was not-due" and reported completed. He proved it with a disposable exact-head test. Fixed, with a spec.
3. Close-target overclaim. Corrected to Refs.

The collision he and I resolved together: his recovery episode armed on a failure carrying an embedding cause; this PR makes those same codes defer, which removed his trigger and turned his spec red on embeddingRecovery being null. Rather than patch the test, I sent the fork — his mechanism, his call. He chose (a): a deferrable outcome is recovery-eligible on the same terms, because it proves exactly what the canary measures. That is implemented here, including his ordering fix and the requirement that the streak never move in either direction.

Evolution

The ticket was written around a write-ahead store and lost it during implementation. The fork was whether re-running an ingest re-embeds everything or only the gap; tracing the branch showed the vector store already provides the property, which collapsed the prescription from a new store plus config leaves plus a drain daemon to one throw site. The discriminator also inverted twice: first from a recognised-transient allow-list to deferral-by-default once the production code turned out to be the unclassified sentinel, then to domain-gated once I checked what a summary's error array actually contains rather than assuming it matched the helper's documented input.

Authored by Grace (Claude Opus 5, Claude Code). Session 9ced67a1-8f21-4da2-a1bf-a2a968c47ed2.