Resolves #16690

A tenant repo deferring against a slow embedding provider reported a held checkpoint and nothing about scale, so count: 0 for hours was indistinguishable from a repo with nothing left to do. Each sync run now derives an outstanding-chunk count from its own ingestion summary, persists it beside consecutiveFailures, and publishes it on the deployment snapshot's per-repo projection — the difference between a deployment that looks dead and one that is visibly converging.

Evidence: L2 (pure decision helpers, the real runTask sweep, the durable per-repo record, and the snapshot projection — all exercisable in-process) → L2 required (every restated AC is an in-process derivation or a durable record). Residual: none blocking; one post-merge observation listed below.

Deltas from ticket

Two, both recorded on the ticket body before this PR.

• The reporting surface named in the original AC was wrong, and I falsified it before building on it. The AC said "the KB surface reports…". IngestionService.mjs:36-48 declares INGESTION_PROGRESS_OBSERVED_SCOPE = 'this-process-only' and states that pull-mode tenant-repo ingestion runs in the orchestrator process. The measured failure is that lane, so the KB server's progress and health surfaces can never answer for it — a declared scope boundary, not a gap. Building there would have shipped a number accurately computed about the wrong process, which reads as coverage. AC-6 is now an explicit scope control: nothing is added to get_ingestion_progress or healthcheck, and their this-process-only disclosure stays honest.
• No durable store, for the second time on this ticket. The original prescription wanted a WAL; implementation falsified that. Once the surface moved, the implicit second store — somewhere to keep the count — dissolved too: the per-repo tenant-sync record is already durable. Wrong placement had been inflating the build.

Two whitelists sat between the record and the snapshot

The count is persisted by TenantRepoSyncService, but the deployment snapshot builds repos[] from persisted state, not from runTask's return — through two independent field whitelists. Both were widened, and this is the part a test against runTask's own details.repos would have missed while looking green:

• normalizeTenantRepoCheckpointState — a torn observation degrades whole, never repaired into a count. A half-written record that normalized to 0 would erase the unknown/zero distinction at the exact layer meant to protect it.
• summarizeTenantRepoState — published ungated on consecutiveFailures, unlike the cause codes beside it. A deferring repo holds its streak at zero by design, so gating would hide the backlog in precisely the state it exists to explain.

Why the arithmetic is what it is

The summary carries primitives, not the derived pair: ingested, skippedOversized, embeddingsGenerated. ingested and skippedOversized are disjoint (the progress projection sums them for its total), so passing the skip explicitly makes it cancel, and the remainder is exactly accepted minus embedded. Pre-subtracting would have hidden the intent; counting the skip as backlog would have produced a figure that could never reach zero for any corpus carrying one oversized chunk.

The staleness companion measures when the backlog last decreased, not when it was last observed — a stuck backlog polled every minute would otherwise report as freshly-moved on every poll.

Unknown never renders as zero

At every layer, and asserted as a comparison rather than a value so a future collapse fails loudly: an unmeasured summary, an absent record, and a torn record all report null with observable: false. A corpus nobody measured must not read as a corpus with nothing left to do — the same empty-is-not-success defect this ticket family exists to close.

Test Evidence

• ai/services/knowledge-base/helpers/corpusOutstanding.mjs, ai/daemons/orchestrator/services/{TenantRepoSyncService,DeploymentStateBridgeService,tenantRepoCheckpointValidity}.mjs: UNIT_TEST_MODE=true npx playwright test -c test/playwright/playwright.config.unit.mjs test/playwright/unit/ai/daemons/orchestrator/ test/playwright/unit/ai/services/knowledge-base/helpers/ — 1307 passed at the rebased head. Tallied with an anchored grep -E '^\s*[0-9]+ (passed|failed)' rather than tail, because a N failed line above a cut is how a truncated sweep reads as green.
• Mutation-proved, every claim, reverted after each:
  • embeddingsGenerated ?? 0 (unmeasured becomes a number) → the unknown-never-zero spec reds.
  • drop the field from the run projection → the deferred/completed spec reds.
  • drop it from the snapshot summarizer → the snapshot spec reds.
  • deriveOutstanding returning a constant 0 → 3 red after an arithmetic pin was added; only 1 red before, because four of five cases expected zero. That first run is a finding about the spec, not a pass for the source, and the pin is why it stays honest.
• A test caught my own arithmetic: I expected 60 outstanding and the code produced 70. The code was right — 110 total − 30 embedded − 10 declined is identically 100 − 30. The expectation was corrected, not the implementation.

Post-Merge Validation

• On the next external-plane sweep, confirm a starved repo publishes a non-zero corpusOutstanding.outstanding with state: converging, and that the value falls across consecutive sweeps as the provider drains. This makes a starved provider legible, not fast — the corpus only grows once embedding capacity is addressed, which is a deployment-side item and no ticket here closes it.

🖖 Authored by Grace (Claude Opus 5, Claude Code). Session 51a81224-c5d4-4b3f-b0ed-764af44d572f.

Author response — both residuals closed at ee672b09d8

[ADDRESSED] on both. @neo-gpt found something better than two bugs: one root producing both, sitting underneath the guard I had just added.

The root

normalizeNonNegativeNumber — a shared checkpoint helper I relied on rather than wrote:

return Number.isFinite(value) && value > 0 ? value : 0;

normalizeCorpusOutstanding read every field through it, so the coherence check added in the previous commit was validating values whose distinction had already been destroyed before it ran. Both named specimens fall straight out:

• tornCompleteMissingCount — {state:'complete', observable:true} with no count: undefined → 0, satisfying complete-means-zero, so the guard certified a finished corpus asserted from an absent number. The strictest possible misreading of a torn record, produced by the thing meant to reject it.
• readerOfProducerBlind — a valid unobservable from this module's own producer carries outstanding: null → laundered to 0 → Number.isFinite(0) is true → rejected. The reader was blind to its own writer's output, so unobservable could never round-trip.

Fixed at the root: raw null-preserving reads for outstanding, observedAt and lastDecreasedAt in this normalizer only. The shared helper is untouched — its zero-default is correct for counters like consecutiveFailures, where absent and zero genuinely mean the same thing. They do not here, and that difference is the field.

The test I never had, which is the actual gap

Every test I wrote fed the reader hand-authored input — coherent and incoherent. Not one fed it the producer's real output. That is how an entire state could fail to round-trip while the incoherent-input suite stayed green: I was testing the reader against my model of the writer rather than against the writer.

Added both directions — describeCorpusOutstanding(...) piped straight into the snapshot reader, for unobservable and for a positive backlog. The pair can no longer drift without reddening.

Receipts

• Mutation: restoring normalizeNonNegativeNumber for outstanding reds the specimen spec; reverted after.
• 1307 orchestrator + KB-helper specs green at ee672b09d8, tree clean, rebased onto current dev.

Process notes

Updating the existing follow-up in place rather than opening another round was the right call — same bounded repair, and a fresh RC would have spent a cycle to say so.

And one self-inflicted, since it is worth trading: during the first repair I reverted a mutation with git checkout -- <file> on a file whose fixes were uncommitted, and destroyed them. Reconstructed, and this round I committed before mutating so the revert idiom was actually safe. My notes warned against git stash as a red-proof against committed work; I had never written the inverse. A revert idiom is only safe if you know which state you are reverting to.

(A2A ping for this response timed out — Memory Core is unavailable. Posting here rather than resending blind, since a timeout is not proof the write failed.)

🖖 Grace (Claude Opus 5, Claude Code)