LearnNewsExamplesServices
Frontmatter
id17129
titleA deterministically undeliverable chunk blocks every chunk and repo behind it forever — ceiling-fired-twice must classify as undeliverable and skip with a receipt
stateClosed
labels
bugaiagent-os
assigneesneo-opus-vega
createdAtAug 14, 2026, 7:06 PM
updatedAtAug 14, 2026, 11:26 PM
githubUrlhttps://github.com/neomjs/neo/issues/17129
authorneo-opus-vega
commentsCount1
parentIssue17072
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 14, 2026, 11:26 PM

A deterministically undeliverable chunk blocks every chunk and repo behind it forever — ceiling-fired-twice must classify as undeliverable and skip with a receipt

Closed Backlog/active-chunk-16 bugaiagent-os
neo-opus-vega
neo-opus-vega commented on Aug 14, 2026, 7:06 PM

Problem

Three head-of-line blockings compose into "one oversized file disables the entire knowledge base":

  1. Admission checks slot fit (safeProcessingLimitTokens), never deliverability inside the enforced call ceiling — so a chunk whose intrinsic service time exceeds batchEmbeddingTimeoutMs is dispatched anyway.
  2. The provider-timeout classification ends the whole sweep (correct against queueing behind still-running work) — but with in-order re-selection the undeliverable chunk is always first, so every chunk behind it is never attempted. The retry-exhaustion path already carries skip-and-continue protection against exactly this shape (its comment: "Nothing after it is ever attempted again"); the timeout path overrides it.
  3. The repo's sweep never completes, so sibling repos never get a turn (observed live: three repos starved for hours behind one chunk that needs ~35-40 minutes against a 30-minute ceiling).

Client-visible result on a constrained plane: "KB ingestion does not work; N cores burn for hours with zero progress" — while every instrument reports handled failures.

Acceptance Criteria

  1. A chunk whose embed call ends in a timeout-class failure on a configurable number of consecutive exactly-attributed attempts (default 2, leaf-owned per ADR-0019) is classified undeliverable-at-geometry — a typed disposition with a receipt (chunk id, token estimate, attempts, effective ceiling), reusing the existing skip/poison persistence machinery. It is excluded from re-selection until the geometry, ceiling, or content changes (content-hash keyed). (Amended in Round-2 from "at the batch head": a timeout from a multi-input provider request names the request, not a member — blaming the head strikes innocents. Exact attribution = the failed request held exactly one input, proven by a single-chunk dispatch or the transport's failedTextOffset/failedTextCount producer span; a multi-input timeout only marks its members as isolation suspects, which are then dispatched alone to earn exact evidence.)
  2. Graduation excises the chunk from all FUTURE dispatch (the persisted disposition filters it out of batch assembly); the graduating sweep itself still ends per the abandoned-work rule — dispatching the remainder immediately would queue fresh provider work behind the just-abandoned headless grind, the exact hazard the end-sweep semantics exist to prevent. The NEXT sweep proceeds past the excised chunk on an idle engine; a known-undeliverable chunk never re-dispatches to the provider. (Amended pre-implementation from "the same sweep continues": immediate continuation would recreate the abandoned-work queueing hazard.)
  3. The ingestion/sync report and the deployment-state surface carry the undeliverable census (count + ids) so a plane owner sees "N documents deferred pending chunking/geometry" instead of silence. A run whose only remaining error rows are the durable fence classifies complete (checkpoint advances, streak resets) — the census, not a perpetually held checkpoint, is what keeps the fenced chunks visible.
  4. Production-path spec driven through VectorService.embed() with the real poison store: a corpus of [typical-A, monster, typical-B] under a multi-input transport shape graduates ONLY the monster with the receipt, persists both typicals, never re-dispatches the fenced chunk, and the repo attempt returns/classifies complete so sibling-repo rotation is unblocked. Sweep budget: two exact strikes (two sweeps) under single-input geometry; one suspicion sweep plus two exact-strike isolation sweeps under multi-input geometry. (Amended in Round-2 from "completes every typical chunk across two sweeps ... receipt on attempt 2": the two-sweep arithmetic was only true of single-input geometry, and the original spec drove embedChunks() directly with the presumed monster at index 0 — mutation-insensitive to the head-blame attribution defect.)
  5. Relationship boundaries: admission-side prevention (dispatching nothing undeliverable) remains #17113's serviceability contract; content-side repair (making big files embeddable) remains the parser work. This ticket owns only the fail-forward semantics.

Evidence class

Live constrained-plane observation 2026-08-14 (~15:13-16:56Z): head chunk ceiling-cancelled at 35:01 engine-uptime with memory stable — intrinsic rate <14 tok/s vs a 30-min ceiling; corpus frozen at 111 for over an hour while sweeps fired every 60s; three sibling repos starved. Source-bound to the timeout classification and retry-exhaustion skip path in VectorService.embedChunks.

Part of epic #17072.

Contract Ledger

Target surface Source of authority Behavior Failure / fallback Evidence
kb.undeliverableTimeoutStrikes leaf (NEO_KB_EMBEDDING_UNDELIVERABLE_TIMEOUT_STRIKES, default 2) ai/mcp/server/knowledge-base/configBase.mjs + parity snapshot Consecutive exactly-attributed single-input timeout-class failures for the same chunk before graduation Absent env = default 2; non-positive values rejected by leaf type Config-leaf-parity gate + unit arm
KB_VECTOR_EMBED_UNDELIVERABLE_AT_GEOMETRY bounded reason code embedFailureClassification.mjs registry Names the graduated disposition in poison-store rows and reports Unknown codes remain rejected by the store's closed-schema validation Store round-trip spec
Poison-store generation input gains the effective embed call ceiling VectorService.resolveEmbeddingPoisonGeneration() + createEmbeddingGenerationId optional fifth coordinate A ceiling change invalidates ALL suppression evidence (undeliverable AND content-poison) so raised ceilings re-offer previously undeliverable chunks Cost: one isolation re-proof cycle for prior content-poison after a ceiling change — correctness over thrift Generation-invalidation spec arm through the production embed() path (the Round-1 head carried the ceiling on the coordinate object while the hash silently dropped it — presence without the property)
Strike/suspicion evidence generation-scoped process-local automaton in VectorService Strikes: consecutive per chunk, exact single-input attribution only, reset on any dispatched non-timeout provider outcome, overlap-guarded so concurrent attempts cannot fabricate a sequential pair. Suspects: members of a timed-out multi-input request, dispatched alone until cleared or graduated Process restart resets evidence: a chunk pays its strikes again after a crash — bounded extra cost, no correctness loss. Generation change resets all transient evidence Automaton spec arms (attribution, reset, overlap, generation)
Undeliverable census ({count, ids}) ingest summary fence rows → TenantRepoSyncService checkpoint → tenantRepoCheckpointValidity normalizer → DeploymentStateBridgeService snapshot AC-3 operator surface: persisted on deferred AND completed checkpoints, projected unconditionally beside corpusOutstanding Fail-closed at both reader boundaries: a torn census degrades whole to null (unobserved), never to a smaller census Sync-composition arm + normalizer torn-shape arms + snapshot projection arm
tobiu referenced in commit 233df4c - "fix(ai): a twice-expired call ceiling graduates the chunk to undeliverable-at-geometry (#17129) (#17133) on Aug 14, 2026, 11:26 PM
tobiu closed this issue on Aug 14, 2026, 11:26 PM