Provider-activity in-flight records never expire, so abandoned work permanently inflates admission counts

Context

External-plane snapshot, 2026-08-13 19:59Z, providerActivity from the orchestrator deployment-state bridge. totalInFlight: 26. The oldest entries:

{"service":"memory-core","operationStage":"embedding-canary","provider":"ollama",
 "enqueuedAt":"2026-08-12T08:20:48.562Z","startedAt":"2026-08-12T08:20:48.562Z","elapsedMs":128356357}
{"service":"knowledge-base","operationStage":"kb-tenant-ingestion-embedding","provider":"ollama",
 "enqueuedAt":"2026-08-12T09:30:04.928Z","elapsedMs":124199991}
{"service":"unknown","operationStage":"unknown","provider":"ollama",
 "enqueuedAt":"2026-08-12T20:01:44.263Z","startedAt":null,"elapsedMs":86300656}

35.6 hours, 34.5 hours, 24 hours — and several were never dispatched at all (startedAt: null).

That plane stopped using Ollama for embeddings at 2026-08-13T07:21:55 (last lastSeenAt on every ollama embedding aggregate). Twelve hours later the ledger still reports:

"knowledge-base::ollama": {"executing": 13, "waiting": 0}
"unknown::ollama":       {"executing":  1, "waiting": 4}

Thirteen "executing" embeddings on a provider that has had no traffic for twelve hours.

The Problem

ai/services/shared/providerActivityLedger.mjs records an activity on dispatch and clears it on settle. When a request neither settles nor is cancelled in a way the ledger observes — client abort, provider timeout that strands work (#16853), or a process boundary crossed mid-flight — the record is never released.

Those records are not merely cosmetic: nativeAdmission derives executing / waiting counts from them, and admission decisions read those counts. A permanently-occupied slot is indistinguishable from a busy one, so the accounting degrades monotonically over the life of the process and can only be cleared by a restart.

Two consequences observed together on the same plane:

1. False saturation. A lane can be reported as fully executing while no real work exists on it.
2. Diagnostic corruption. service: "unknown" and operationStage: "unknown" entries accumulate, so the ledger — the instrument used to diagnose exactly this class of incident — becomes progressively less trustworthy the longer the incident runs.

The ledger already carries everything needed to detect this: enqueuedAt, startedAt, and elapsedMs. Nothing consumes them for expiry.

Architectural Reality

• Provider deadlines are bounded (300000ms for embeddings on the observed plane). Any in-flight record older than its own class deadline by a wide margin is definitionally leaked, because no legitimate request outlives its deadline by hours.
• startedAt: null with a 24-hour enqueuedAt is a stronger signal still: the item was never dispatched, so nothing can settle it.
• This is distinct from #16853. That ticket concerns provider-side work surviving a client disconnect (a real runner still burning CPU). This is Neo's own bookkeeping outliving both the request and the provider.

The Fix (shape)

1. Expire in-flight records past a bound derived from the item's own deadline class, rather than a global constant, so a legitimately long batch is never reaped early.
2. Reap on read as well as on a timer, so a snapshot never reports a count the ledger itself can see is stale.
3. Report reaped entries with a distinguishing disposition (abandoned / unsettled) rather than deleting them silently — the count of leaks is itself a health signal, and silent deletion would hide the very defects that cause it.
4. Never let a reaped record be re-counted as a completion; it is neither success nor failure.

Acceptance Criteria

• An in-flight record whose age exceeds its class deadline by a defined factor is reaped and no longer counted in nativeAdmission.executing / waiting.
• A record with startedAt: null older than the queue-wait bound is reaped.
• Reaped records surface with a distinguishing disposition and a count, not by disappearing.
• A legitimately long-running batch inside its deadline is not reaped (negative control).
• Switching a role's provider leaves no residual executing count on the abandoned provider after the reap interval.

Out of Scope

• Provider-side stranded runners (#16853).
• Admission ordering (#17062).
• Any change to deadline values.

Avoided Traps

• "Restart clears it." True and useless: it means every long-lived deployment drifts toward false saturation, and restarting is what this incident class is already doing too much of.
• "Use one global TTL." Would reap legitimate long batches; the bound must derive from the item's own class.
• "Delete silently." The leak count is diagnostic signal; hiding it removes the evidence for the next investigation.

Related

• #16853 — early abort strands provider work (a producer of these leaks)
• #17062 — admission ordering starves ingestion
• #17048 — engine-slot monopoly