LearnNewsExamplesServices
Frontmatter
id17345
titleAn undeliverable chunk is quarantined until the generation changes, so a fix nobody signals never retries it
stateClosed
labels
enhancementaiagent-os
assigneesneo-opus-vega
createdAtAug 18, 2026, 1:12 PM
updatedAtAug 19, 2026, 2:02 PM
githubUrlhttps://github.com/neomjs/neo/issues/17345
authorneo-opus-vega
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 19, 2026, 2:02 PM

An undeliverable chunk is quarantined until the generation changes, so a fix nobody signals never retries it

Closed Backlog/active-chunk-17 enhancementaiagent-os
neo-opus-vega
neo-opus-vega commented on Aug 18, 2026, 1:12 PM

Context

#17129 established the disposition that keeps a corpus moving: a chunk the current geometry cannot deliver graduates to KB_VECTOR_EMBED_UNDELIVERABLE_AT_GEOMETRY, stops being offered, and everything behind it proceeds. That was the right call and it works.

Its release condition is the open question. From the graduation log line (VectorService.mjs:1468):

"it stops being offered until the embedding generation (provider, model, dimension, call ceiling) changes."

So a quarantined chunk returns only when one of four declared inputs changes. That is precise and auditable — and it misses the most likely reason a chunk becomes deliverable again.

The Problem

The commonest repair changes none of the four. #17343 is a live example: the admission band is compared in the wrong unit against the wrong leaf, so chunks between the engine ceiling and the safe band are dispatched whole and refused. Fixing it changes the splitter, not the provider, model, dimension, or call ceiling. Every chunk quarantined before that fix stays quarantined after it — the generation is byte-identical.

The same holds for a parser change that emits smaller units, a per-tenant include manifest that removes a vendor tree, or a chunker threshold adjustment. All of them make previously-undeliverable content deliverable, and none of them is a generation change.

The result is a quarantine that is correct on entry and has no exit for the case that actually occurs. The corpus keeps advancing — #17129's goal is met — but a permanent, invisible hole opens in it, and nothing re-examines the hole because the release condition is watching the wrong variable.

Operator direction (2026-08-18): quarantine an oversized chunk "for e.g. a day, to ensure others can pass." A time-boxed release retries after a repair without requiring anyone to signal that a repair happened — which is the property the generation condition lacks, because the repair usually lives in code the generation does not describe.

The Architectural Reality

  • VectorService.mjs:1450 — strike accrual; undeliverableTimeoutStrikes is the entry threshold.
  • VectorService.mjs:1464onPoisonEntries graduation, receipt shape {chunkId, tokenEstimate, attempts, effectiveCeilingMs}.
  • VectorService.mjs:1468 — the release contract, stated in the log line rather than in a named predicate.
  • The disposition store is shared with content poison, and VectorService.mjs:1011 is explicit that the two assert different things: "a content poison is proven bad content, an undeliverable-at-geometry chunk is healthy content the current geometry cannot deliver." That distinction is exactly why a re-offer policy is safe here and would not be for poison.

Entry stays as-is. This ticket changes only when a chunk leaves.

Drift probe on pickup — 2026-08-19, and it supersedes the prescription below

Authored 2026-08-18; probed before implementing. The prescription drifted, and the replacement is smaller and more correct. Recording the supersession rather than editing it away.

The mechanism this ticket asks for already exists and is pinned shut by a literal.

VectorService.resolveEmbeddingPoisonGeneration() builds the generation from provider, model, vectorDimension, embedCallCeilingMs — and strategyVersion. Its own docblock states the intent:

"…so a provider, model, vector-schema, or input-strategy change invalidates prior poison evidence instead of silently suppressing work under a new route."

But strategyVersion: EMBEDDING_POISON_STRATEGY_VERSION, and VectorService.mjs:157 is:

const EMBEDDING_POISON_STRATEGY_VERSION = 'kb-embedding-input-v1';

A static module literal. So the one coordinate that exists to represent the input strategy never moves when the input strategy changes. createVectorGenerationIdentity already guarantees "any change to ANY field yields a new generationId" — the plumbing is complete and the value is frozen.

So the fix is to DERIVE strategyVersion from the inputs that decide embeddability, not to add a timer. The admission band is now a shared surface (ai/embeddingSafeBand.mjs, added by #17343): resolveEmbeddingAdmissionBand({contextLimitTokens, safeProcessingLimitTokens}) plus EMBEDDING_TOKEN_ESTIMATE_DRIFT_FACTOR. Key strategyVersion on the resolved band and:

  • #17343's repair changes the band ⇒ new generation ⇒ quarantine released by design.
  • A future splitter/band change releases automatically, with no human remembering to bump a literal.
  • A change that does not affect embeddability leaves the band identical ⇒ evidence correctly preserved.

Why the timer prescription below is superseded rather than merely bigger. A time-boxed release re-offers a chunk whether or not anything was repaired — which is why it needed AC-3's capped backoff to bound the waste, and why AC-1 reads "after the configured window" rather than "after the repair". A derived strategyVersion releases on the repair and never otherwise. The timer answers "eventually try again"; the defect is "we cannot tell that the thing that would help has happened."

One Out-of-Scope line below is now partly wrong and stays visible: it says #17343's fix "will not release what is already quarantined." True of the fence as written — and today I established that the tenant chunk-ID hash includes parserVersion (IngestionService.mjs:2287/:2315), so a parser-version bump re-identifies chunks and the fence stops matching. The corpus would therefore move by accident, not by release. That accident is not a reason to close this ticket; it is the reason to fix the release path, because it only works for repairs that happen to bump the version.

Revised acceptance criteria — these replace the list below:

  • strategyVersion is derived from the resolved admission band, not a literal. A band change yields a different generation id.
  • A chunk quarantined as undeliverable-at-geometry under band B is re-offered after the band changes, with no provider/model/dimension/ceiling change — that unchanged tuple is the condition under test, and a fixture that also moves one of the four would release on main too and proves nothing.
  • A band that does not change leaves the generation id and the suppression evidence identical — asserted as a negative control, so the derivation cannot become a catch-all that invalidates evidence on every call.
  • Content-poison dispositions follow the same generation rule they follow today; this changes what "the generation" means, not which dispositions honour it. Asserted, since both families share the store.
  • A mutation restoring the literal fails the band-change assertion and only it.
  • No timer, no backoff schedule, no next-eligible instant. If a permanently undeliverable chunk needs bounded retries later, that is a separate ticket with its own evidence.

The Fix — SUPERSEDED by the drift probe above, retained for the record

  1. Give UNDELIVERABLE_AT_GEOMETRY a time-boxed release alongside the existing generation-change release: whichever comes first. Default on the order of a day, configured by a leaf, not a literal.
  2. On release, the chunk is re-offered exactly once through the ordinary path. If it fails the same way it re-quarantines, with its attempt count preserved so the receipt shows a repeat rather than a fresh discovery.
  3. Apply an increasing interval across successive re-quarantines, bounded by a cap, so a permanently undeliverable chunk costs a bounded number of retries per corpus rather than one per window forever.
  4. Keep the release policy for geometry dispositions only. Content poison keeps today's behaviour — re-offering proven-bad content on a timer is a different and worse idea.
  5. Surface the next-eligible instant on the disposition receipt, so an operator reading deployment state can tell a permanent hole from one awaiting its retry.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
KB_VECTOR_EMBED_UNDELIVERABLE_AT_GEOMETRY release VectorService.mjs:1468 generation-change or elapsed window, whichever first generation-change only when the leaf is unset reason-code docs a splitter fix changes no generation input
quarantine window new config leaf env-bound duration, increasing interval, capped today's indefinite hold config docs operator-directed ~1 day
disposition receipt existing receipt shape carries next-eligible instant + repeat count absent field, no consumer break a permanent hole is currently indistinguishable from a waiting one

Decision Record impact

none — extends #17129's disposition rather than challenging it.

Acceptance Criteria

  • A chunk quarantined as undeliverable-at-geometry is re-offered after the configured window with no generation change.
  • A re-offer that fails identically re-quarantines with its prior attempt count preserved, not reset.
  • Successive re-quarantines back off on an increasing, capped interval.
  • Content-poison dispositions are not re-offered on a timer — asserted by a test, since both live in the same store and a shared release path is the obvious wrong implementation.
  • The disposition receipt names the next-eligible instant.
  • Red-proof: a chunk quarantined under geometry G, with G unchanged and the window elapsed, is re-offered after the change and is not re-offered on main. A fixture that also changes the generation would be released on main too and proves nothing — an unchanged generation is the condition under test.

Out of Scope

  • The entry condition and its strike threshold — #17129 owns entry; #17336 covers the death-class trigger that never graduates at all.
  • Fixing the band that produces these chunks — #17343. This ticket exists precisely because that fix will not release what is already quarantined.
  • Retry-at-identical-size on a deterministic refusal — #16972.
  • Content-poison policy.

Avoided Traps

  • Replacing the generation condition instead of adding to it. Generation-change is a genuine and cheaper signal when it does fire; the defect is that it is the only one.
  • A fixed retry interval with no backoff. A permanently undeliverable chunk would then cost one failed dispatch per window forever, on a lane where a single oversized request can be expensive.
  • Sharing the release path with content poison. They live in one store and assert opposite things; a shared timer would re-offer proven-bad content indefinitely.
  • Re-offering the whole quarantined set at once on release. A corpus with many quarantined chunks would produce a thundering retry against the same provider that refused them.

Related

#17129 (established the disposition and the generation-change release) · #17336 (the death-class entry gap) · #17343 (the band defect whose fix changes no generation input — the motivating case) · #16972 (identical-size retry) · #11735 (vendor never-ingest set)

Live latest-open sweep: latest 6 open checked 2026-08-18T11:11:28Z (immediately pre-create, after filing #17343/#17344 minutes earlier), plus a five-term state=all title sweep on quarantine / expiry / undeliverable / re-offer / retry window. Nearest neighbours #17129 (closed, entry-side) and #17336 (open, entry-side); no ticket covers the release condition. A2A recency scan clean — no in-flight claim on this scope.

Origin Session ID: 9ccc2fa1-8843-4796-8e85-5e151c0392d2

Retrieval Hint: query_raw_memories("undeliverable at geometry quarantine release generation-change vs time-boxed window re-offer")

tobiu referenced in commit a126da5 - "fix(ai): the poison generation tracks the band it is fencing against (#17345) (#17382) on Aug 19, 2026, 2:02 PM
tobiu closed this issue on Aug 19, 2026, 2:02 PM