Resumed repo slices forget previously settled embeddings

Context

Recovered while auditing a superseded implementation of #17132 after Grace's merged PR #17299. The slice budget and repo rotation are shipped; the resumed-slice accounting property is not.

Current-source revalidation on 2026-08-21 at origin/dev@3809616cdc still establishes the mismatch:

• IngestionService.mjs:331 keeps summary.ingested = embeddableChunks.length: the accepted embeddable corpus presented on this sweep.
• The incremental VectorService.embed() return still exposes only current-call embedded; the no-op path returns embedded: 0 even when every accepted id is already durable.
• TenantRepoSyncService.buildCorpusOutstandingObservation() still derives outstanding from accepted minus that current-call count.
• PR #17441 changed materialization-proof eligibility for already-classified durable fences; it did not add cumulative settlement. Those closed fences now make it explicit that “settled” is broader than “new vector landed.”
• PR #17456 changed lease-cause exit semantics and the deployment guide, but left this accounting equation intact.

A resumed call can therefore forget durable IDs landed by earlier slices. This is a source-proven arithmetic defect; the required production-composition red below owns its behavioral proof.

Live latest-open sweep: checked the latest 20 open issues at 2026-08-20T21:53:24.789Z; no equivalent found. Recent A2A claim sweep: last 30 messages, no overlapping claim. Semantic KB and live GitHub searches returned no equivalent ticket.

The Problem

For a 100-chunk accepted corpus:

1. Slice 1 lands 20; the current arithmetic reports 100 - 20 = 80 outstanding.
2. Slice 2 re-enters with the same accepted corpus. VectorService internally excludes the 20 durable IDs, but returns only the 20 newly landed in slice 2.
3. The caller again reports 100 - 20 = 80.

The corpus advances while the operator-visible remainder stands still. Different per-slice throughput can even make the number move independently of durable progress.

This is not permission to redefine ingested. #17132's amended AC-5 correctly preserves ingested = accepted; collapsing it to landed makes the existing outstanding equation identically zero. The missing terms are authoritative cumulative settled and remaining.

The Architectural Reality

• Incremental VectorService.embed() already owns the full decision set: unique accepted ids, corpus-scoped durable ids, known poison/fence ids, newly landed ids, terminal skips, retryable failed batches, and the yielded remainder.
• Shadow-swap already folds alreadyEmbedded into its current embedded result; incremental mode does not. Both strategies can derive the same post-call settlement tuple without a second collection scan.
• IngestionService owns the consumed summary contract and currently exposes accepted ingested, newly landed embeddingsGenerated, and yielded.
• TenantRepoSyncService persists corpusOutstanding; DeploymentStateBridgeService already projects that object unchanged into each repo row.
• Closed durable fences and terminal guardrail skips are settled for the current generation because an identical next sweep will not offer them to the provider. Retryable failed batches and yielded/undispatched ids remain.
• The fix belongs in the existing Vector → Ingestion → TenantRepo result chain. No new persistence store or competing authority is needed; the existing checkpoint object receives additive observation fields.

The Fix

1. Define one algebra over the unique accepted embeddable ids for a group: settled + remaining === accepted. remaining is exactly the id set an identical next sweep would still offer to provider work; settled is the complement.
2. Return exact non-negative settled and remaining from both VectorService strategies. Durable vector rows, already-classified closed fences, and terminal skips contribute to settled; retryable failures plus yielded/undispatched ids contribute to remaining.
3. Propagate those fields through the Ingestion summary without changing ingested = accepted or embeddingsGenerated = newly landed. Aggregate only when every repo group supplied a coherent tuple. An absent legacy producer leaves both values unobserved; a present malformed or inconsistent tuple fails loud.
4. Build corpusOutstanding from authoritative summary.remaining. Retain outstanding as the backward-compatible alias equal to remaining, and add cumulative settled / remaining to the same persisted observation object.
5. Project that existing object on partial-progress and completed repo rows, update OpenAPI/JSDoc/cloud documentation, and keep lease-vs-slice cause work in #17398/#17414.

Contract Ledger Matrix

Decision Record impact: none — this repairs accounting inside existing result and deployment-observation contracts.

Acceptance Criteria

• Against current dev, a production-composition witness with four due repos at concurrencyLimit=2 proves repo A resumes across two slices from durable landed ids and currently reports a non-decreasing or stale remainder.
• After repair, A's remaining strictly decreases when the second clean slice lands new ids, and reaches zero only when the accepted corpus is settled.
• A no-op resume over an already-settled corpus reports settled = ingested and remaining = 0 from durable ids, not from a current-call zero.
• Incremental and shadow-swap paths use the same settled / remaining meanings.
• Closed durable fences and terminal skips are settled; retryable failed batches and yielded/undispatched ids remain. Changing fence eligibility itself stays out of scope.
• summary.ingested remains the accepted term and summary.embeddingsGenerated the newly-landed term; mutations redefining either or violating settled + remaining === ingested turn guards red.
• Across multiple groups, one missing producer makes the aggregate unobserved; missing, fractional, negative, or internally inconsistent present counts cannot project zero outstanding.
• corpusOutstanding.outstanding === corpusOutstanding.remaining on observed rows, with cumulative settled beside it; unobserved rows carry null counts.
• The Ingestion/OpenAPI/JSDoc and deployment-guide contracts match the shipped fields.
• No new persistence store or competing authority is introduced; only additive fields on the existing checkpoint observation are allowed.

Out of Scope

• Lease-vs-slice cause and exit semantics: #17398 and #17414.
• Changing which errors qualify as durable-fence receipts: #17139/#17440 lineage. This ticket only accounts for the already-classified closed rows.
• Changing sliceBudgetMs, semaphore admission, or provider batching.
• Redefining ingested as landed or embeddingsGenerated as cumulative.
• A new durable store or a second settlement authority.

Avoided Traps

• Subtracting current work from lifetime work. A per-call delta is not cumulative settlement.
• Crediting the whole group on absent data. Under-counting re-offers work; over-counting suppresses it forever.
• Porting a stale source blob. The recovered implementation predates later Vector, tenant-sync, and checkpoint changes; only the property and focused arms are reusable.

Related

• #17132 / PR #17299 — shipped slice fairness; this is a post-merge accounting follow-up.
• #17410 — closed live observation that outstanding did not move; its checkpoint-deadlock premise was withdrawn, but the symptom matches this arithmetic.
• #17398 / #17414 — distinct cause-bearing lease/slice work.
• PR #17441 — durable-fence completion changed the terminal set this accounting must classify.
• PR #17456 — lease-caused exit semantics landed after ticket creation; it does not solve cumulative settlement.

Origin Session ID: 5227e910-6f64-46da-a0d1-d6fb1cdfbb9f

Retrieval Hint: query_raw_memories("resumed tenant repo slice settled remaining outstanding accounting"); source anchors IngestionService.mjs:330, VectorService.mjs:2622, TenantRepoSyncService.buildCorpusOutstandingObservation.