LearnNewsExamplesServices
Frontmatter
id16564
titleA deferral streak is unmeasurable: deferredAt restarts on every poll
stateClosed
labels
bugai
assignees[]
createdAtAug 5, 2026, 9:05 PM
updatedAtAug 5, 2026, 10:28 PM
githubUrlhttps://github.com/neomjs/neo/issues/16564
authorneo-opus-vega
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 5, 2026, 10:28 PM

A deferral streak is unmeasurable: deferredAt restarts on every poll

neo-opus-vega
neo-opus-vega commented on Aug 5, 2026, 9:05 PM

Context

Split out of #16561 as its delivered leaf. That ticket owns the starvation (backup unable to run) and its repair; this one owns the narrower, already-implemented half: the starvation was unmeasurable, so nothing could have reported it.

Measured on the live plane 2026-08-05T18:1xZ, on an otherwise healthy deployment: the last backup was backup-2026-08-05T09-39-56.157Z, 8.5 hours earlier, with tenant-repo-sync holding the heavy-maintenance lease. The plane reported healthy throughout.

The Problem

MaintenanceBackpressureService.recordDeferral stamps deferredAt: new Date().toISOString() on every deferral outcome. That field answers "when was the most recent deferral". The consumer needs "how long has this task been unable to run".

So on every poll the backup truthfully reported a deferral seconds old, and no surface anywhere accumulated them. grep for deferredSince / deferralCount / consecutiveDefer across ai/daemons/orchestrator/ returned nothing (positive control: picker.mjs and remConsolidationLivenessWatchdog.mjs both match on starv, so matcher and scope work).

The keying is the load-bearing detail. The obvious place to hang streak state is dedupKey, which already exists for log dedup — but it embeds the holder (<task>:lease-held-by-<owner>). A streak keyed on it restarts whenever the blocker rotates, and rotation is exactly what happened: tenant-repo-sync → summary → tenant-repo-sync, three "fresh" deferrals covering one continuous 8.5-hour starvation. The stable key is the starved task, not the thing blocking it.

Generalises past leases: any duration metric keyed on the cause rather than the victim resets precisely when the problem is worst. (@neo-opus-grace's framing, from the #16562 review.)

The Architectural Reality

  • ai/daemons/orchestrator/services/MaintenanceBackpressureService.mjsrecordDeferral (per-poll deferredAt, no accumulation) and clearDeferralLogState, which already marks the exact streak boundary: a task that ran is a task that was not starved. Called from the successful-run paths, so the lifecycle to reuse exists.
  • RECOGNIZED_DEFERRAL_REASON_CODES (scheduling/pipeline.mjs:51-57) — deferrals are already first-class with reason codes, so this is an extension of a structured seam, not a new concept.
  • deferralLogKeys — the per-blocker dedup Set whose keying must not be reused for the streak, per above.

The Fix

Implemented in PR #16562:

  1. deferralStreakStarts (taskName -> ISO), stamped on a task's first deferral, emitted as deferredSince beside the existing deferredAt.
  2. Ended by clearDeferralLogState, reusing the existing boundary rather than adding a second lifecycle.
  3. An absent map emits no deferredSince rather than stamping now — a falsely-fresh streak is worse than a missing field, because a consumer reading a just-started streak concludes the task is fine.

Contract Ledger Matrix

Target Surface Source of Authority Behavior Fallback / Error Semantics
recordDeferral skipped-outcome payload this ticket gains deferredSince alongside deferredAt absent streak map ⇒ field omitted, never stamped now
clearDeferralLogState existing additionally ends the streak no map ⇒ no-op, unchanged
deferralLogKeys keying existing unchanged — stays per-blocker for log dedup
scheduling / PRIORITY_ZERO_TASKS pipeline.mjs:13 unchanged

Decision Record impact

none. Extends an existing return payload; no ADR governs deferral telemetry.

Acceptance Criteria

  • A deferred task's outcome carries the start of its current unbroken deferral streak, distinct from the per-poll timestamp.
  • The streak survives a change of blocker, with controls proving the blocker actually rotated and that the dedup keys diverged.
  • Running the task ends the streak, so a resolved starvation stops reporting.
  • Clearing one task's streak does not clear a co-starved sibling's (positive control in the same assertion).
  • An absent streak map omits the field rather than emitting a falsely-fresh value.
  • Mutation-proven: keying the streak on dedupKey instead of taskName fails the suite — the mutation reproduces the original defect rather than merely breaking a test.

Out of Scope

  • Making the backup actually run. #16561 owns the starvation and its repair — now scoped to adding shouldYield polling to TenantRepoSyncService and summarize-sessions, the two holders that do not poll the existing cooperative-yield primitive.
  • Changing scheduling. backup is already PRIORITY_ZERO_TASKS (@neo-opus-grace's June fix, c8e010d7de); the cadence-normalisation hypothesis is falsified on #16561.
  • Lowering staleAfterMs. It is the crash-recovery backstop, not the fairness lever.
  • Health-surface thresholds. Reporting a starved task as degraded is a consumer of this field; this ticket produces the field.

Avoided Traps

  • Keying the streak on dedupKey because it already exists — reproduces the exact invisibility this fixes.
  • Stamping now when no streak state is available — a falsely-fresh streak reads as health.
  • Asserting timestamp inequality across a clear + re-defer. Two deferrals inside one millisecond produce identical toISOString() values; that assertion tests clock resolution, not the property. Re-asserted as a state transition.

Related

  • #16561 — the starvation itself and its repair (parent concern; this is its measurable half).
  • #16563 — backup content integrity; disjoint from backup scheduling.
  • #16557 — a ~2-minute ingest collapses the lease symptom without making a 6-hour staleAfterMs correct.

Live latest-open sweep: latest 12 open issues checked immediately before filing; #16561 is the parent concern being split, #16563 is content-not-scheduling, neither covers deferral telemetry. A2A in-flight claim sweep: no competing claim — this is a split of my own ticket at a reviewer's request.

Origin Session ID: 11695cce-9854-4be2-80c3-8ea4322298bf

Retrieval Hint: query_raw_memories("deferral streak deferredSince keyed on starved task not dedupKey holder rotation invisible starvation")

tobiu closed this issue on Aug 5, 2026, 10:28 PM