Problem
NEO_KB_EMBEDDING_BATCH_SIZE (default 5) is the durable unit of KB ingestion: VectorService/IngestionService persist embeddings only after the full slice returns. One slow or failed chunk therefore discards up to four completed provider results, and the retry re-submits identical work.
On a constrained CPU-only plane this composes into a zero-progress loop: a slot-legal 9,144-token chunk was measured completing cleanly (~6 min at ~26 tok/s) and its result — plus its completed slice siblings — never persisted, because a caller-side giveup fired first. The engine then recomputes the identical chunks on every sweep: full CPU burn, zero corpus growth. Work the provider finished must never be paid for twice.
Related but distinct: #16972 (timeout retries at identical batch size — convergence), #16853 (abort strand class). This ticket owns work conservation.
Acceptance Criteria
- No completed embedding is ever re-purchased across any non-crash exit of the embed call (success, yield, failure, timeout): the completed prefix is durably persisted before the error propagates — with the prefix WRITE riding the shared retry budget (a transient storage failure retries the write, never re-enters the provider, and never escapes retry accounting) — and a re-sweep re-submits only chunks without persisted vectors. (Truth-folded 2026-08-14 from literal per-chunk-before-next-dispatch persistence — the original is preserved in the correction comment; process-crash residue is explicitly out of scope.)
- A re-sweep after a mid-slice failure re-submits only chunks without persisted vectors — proven by a production-path spec (real
VectorService + ingestion collaborators, temp state), not mock seams.
- The batch retry ladder and dedupe/upsert semantics are regression-covered at the new persistence boundary.
NEO_KB_EMBEDDING_BATCH_SIZE=1 remains a valid deployment-side containment (slice == chunk) and is documented as such at the leaf.
Evidence class
Live engine task log + tenant-sync state on a constrained CPU-only plane, 2026-08-14 (completed n_tokens=9144 task with zero corpus delta); source-bound to the slice persistence path.
Part of epic #17072.
Problem
NEO_KB_EMBEDDING_BATCH_SIZE(default 5) is the durable unit of KB ingestion:VectorService/IngestionServicepersist embeddings only after the full slice returns. One slow or failed chunk therefore discards up to four completed provider results, and the retry re-submits identical work.On a constrained CPU-only plane this composes into a zero-progress loop: a slot-legal 9,144-token chunk was measured completing cleanly (~6 min at ~26 tok/s) and its result — plus its completed slice siblings — never persisted, because a caller-side giveup fired first. The engine then recomputes the identical chunks on every sweep: full CPU burn, zero corpus growth. Work the provider finished must never be paid for twice.
Related but distinct: #16972 (timeout retries at identical batch size — convergence), #16853 (abort strand class). This ticket owns work conservation.
Acceptance Criteria
VectorService+ ingestion collaborators, temp state), not mock seams.NEO_KB_EMBEDDING_BATCH_SIZE=1remains a valid deployment-side containment (slice == chunk) and is documented as such at the leaf.Evidence class
Live engine task log + tenant-sync state on a constrained CPU-only plane, 2026-08-14 (completed n_tokens=9144 task with zero corpus delta); source-bound to the slice persistence path.
Part of epic #17072.