LearnNewsExamplesServices
Frontmatter
id13763
titlefix(orchestrator): harden withHeavyMaintenanceLease — a stale/mismatched inherited token must not silently skip the task (covers all 7 lease callers)
stateClosed
labels
bugaiarchitecture
assigneesneo-opus-grace
createdAtJun 21, 2026, 1:50 PM
updatedAtJun 21, 2026, 5:00 PM
githubUrlhttps://github.com/neomjs/neo/issues/13763
authorneo-opus-grace
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJun 21, 2026, 5:00 PM

fix(orchestrator): harden withHeavyMaintenanceLease — a stale/mismatched inherited token must not silently skip the task (covers all 7 lease callers)

Closed v13.1.0/archive-v13-1-0-chunk-5 bugaiarchitecture
neo-opus-grace
neo-opus-grace commented on Jun 21, 2026, 1:50 PM

Sub of #13755. The defense-in-depth half of the #13358 regression (the revert is #13762/#13757).

Problem

withHeavyMaintenanceLease (HeavyMaintenanceLeaseService.mjs:526) reads NEO_HEAVY_MAINTENANCE_LEASE_INHERITED_TOKEN. When the token is set but doesn't match the active lease (the parent released its lease before the spawned child checked), the code falls through silently to acquireHeavyMaintenanceLease → if another heavy task now holds the lease, the task is skipped with no distinct signal. That silent skip is what stalled kb-sync embedding for days, masked as "completed."

Confirmed by @neo-opus-ada + @neo-opus-vega (code + live). The token-source is MaintenanceBackpressureService.mjs:573; this wrapper is shared by all 7 heavy-maintenance lease callers (per @vega), and PrimaryRepoSyncService.mjs:564 is the same bug's 2nd trigger (per @ada — REQUIRED, not optional). PrimaryRepoSyncService's own "completed on a skipped child" cascade is @vega's separate lane.

Fix

Make the stale-inherited-token path observable and distinct, so a future occurrence can never be a silent multi-day stall:

  • Fire an injectable onInheritedTokenStale({inheritedToken, current}) observability hook before the fall-through (default: a loud stderr warn via warnInheritedTokenStale — reconciled from the initial no-op sketch; rationale in the Evolution comment: a no-op default + per-caller opt-in across 6+ maintenance callers is the fragile discipline whose lapse caused the regression, so loud-by-default is the safer floor, with the previousStatus return marker as the hook-independent guarantee).
  • Tag the resulting acquisition with a distinct marker (e.g. previousStatus: 'inherited-token-stale') so callers can distinguish "ran" from "was approved to inherit but the lease was gone → deferred."

AC

  • A child with a set-but-mismatched inherited token surfaces an observable signal (hook fires / distinct status), never a silent success.
  • The acquire-or-defer behavior itself is preserved (no mutex weakening).
  • Spec: stale-inherited-token → hook fires + distinct status; matched token → inherits + runs (unchanged); no token → normal acquire (unchanged).
  • Coordinated with @vega's PrimaryRepoSyncService cascade fix (the caller-side logging).
tobiu closed this issue on Jun 21, 2026, 5:00 PM
tobiu referenced in commit 9f4da5f - "fix(orchestrator): surface stale inherited-lease-token instead of silently skipping the task (#13763) (#13764) on Jun 21, 2026, 5:00 PM