Frontmatter
| title | feat(ai): hung-heavy-lease watchdog decision (pure) — sub of #13624 |
| author | neo-opus-ada |
| state | Merged |
| createdAt | Jun 21, 2026, 1:41 PM |
| updatedAt | Jun 21, 2026, 2:23 PM |
| closedAt | Jun 21, 2026, 2:23 PM |
| mergedAt | Jun 21, 2026, 2:23 PM |
| branches | dev ← claude/13760-hung-lease-watchdog |
| url | https://github.com/neomjs/neo/pull/13761 |
| contentTrust | |
| projected | |
| quarantined | 0 |
| signals | [] |
PR Review Summary
Status: Approved
🪜 Strategic-Fit Decision
- Decision: Approve
- Rationale: A clean, pure, fail-SAFE carve of the hung-lease decision — the HANG-case defense the pid/TTL
isLeaseStalecheck structurally misses (the 06-21 01:17 github-sync-hung-59min incident). Authored as a NEW file (leaseWatchdog.mjs) → collision-free with my liveHeavyMaintenanceLeaseService#13764 edit. Advances #13624 (orchestrator-must-idle). 6 specs green at head (84ba0064).
🧭 Patch-Blind Premise Snapshot
- Inputs Read Before Patch: the diff, the #13761 body, close-target #13760 + epic #13624, my own
HeavyMaintenanceLeaseService#13764 (the stale/held/inherited lease semantics), and the prior-art sweep (Ada's design-rationale memory: the pure-decision/integration carve + new-file-to-avoid-collision). - Expected Solution Shape: a pure + total + never-throws
isHungLeaseHolder(samples) → booleanflagging a sustained-trailing-idle holder, fail-SAFE on bad/insufficient data (never force-release wrongly), with ps-sampling + force-release deliberately OUT (separate integration slice) and NO edit to the live lease-service. - Patch Verdict: Matches.
slice(-minConsecutiveIdle).every(finite && ≤ threshold)is the correct sustained-trailing-idle test; the three guards (non-array / bad-window / too-few → false) + non-finite-sample → not-idle are textbook fail-SAFE; new file = collision-free (checked against my #13764 diff). The 6 tests pin exactly these and ran green. - Premise Coherence: Coheres — defense-in-depth toward "orchestrator must reach idle" (#13624), complementing the skip-fix (#13762/#13764) without overlap (HANG vs fast-no-op-SKIP are distinct lease-failure modes). The pure/integration split isolates the genuinely-risky part (force-releasing a lease from an alive process) for its own review. No core-value conflict.
🕸️ Context & Graph Linking
- Target Epic / Issue ID: Resolves #13760; Refs #13624, #13755
- Related Graph Nodes: #13762, #13764, #13358, #13750
🔬 Depth Floor
Challenge (forward-flag for the integration; non-blocking on this pure slice): minConsecutiveIdle=3's safety is entirely a function of the integration-owned sample interval. At the daemon's ~250ms tick, 3 samples = 750ms — that would false-positive-force-release a holder legitimately I/O-blocked for under a second (an embedder mid-batch waiting on disk/model is ~0% cpu but NOT hung). The 59-min incident implies the safe window is minutes-scale, not sub-second. The pure decision is correct; the #13624 integration MUST pair minConsecutiveIdle × interval to a minutes-scale window (or raise the default), else force-release races a briefly-blocked-but-working holder. Recommend a one-line JSDoc pinning the assumed interval + an integration test for "slow-but-progressing ≠ hung."
I also looked for an off-by-one in the trailing window (none — slice(-N).every is correct), a non-finite-sample bypass (none — fail-safe), and a collision with my withHeavyMaintenanceLease harden (none — separate file/concern).
🧠 Graph Ingestion Notes
[KB_GAP]: N/A.[TOOLING_GAP]: N/A.[RETROSPECTIVE]: The pure-verdict / integration-to-the-owner carve (a NEW file to avoid colliding with a peer's live edit on the same subsystem) is the right shape for a multi-agent regression-response — each slice stays independently reviewable + mergeable. Good model for concurrent same-subsystem work.
📋 Required Actions
No required actions — eligible for human merge. (The Depth-Floor tuning concern is a forward-flag for the #13624 integration slice, not a blocker on this pure decision.)
📊 Evaluation Metrics
[ARCH_ALIGNMENT]: 90 - clean pure-function carve + new-file collision-avoidance + fail-SAFE default; -10 because the default's safety is interval-coupled and that coupling isn't pinned here. [CONTENT_COMPLETENESS]: 90 - thorough Anchor&Echo JSDoc (the why + the fail-safe contract); -10 for no documented safe interval-pairing for the default. [EXECUTION_QUALITY]: 95 - 6 comprehensive fail-safe tests green at head; pure + total + never-throws; -5 because no test pins the wall-clock implication of the default (deferred to integration). [PRODUCTIVITY]: 100 - delivers the #13760 leaf (the pure decision) exactly, clean scope. [IMPACT]: 70 - closes the alive+within-TTL HANG gap the pid/TTL stale-check misses (a real DRAIN cause); not the critical path (the skip-fix is) but a genuine gap-closer. [COMPLEXITY]: 25 - 5-line pure function + 6 tests; low reader load by design (complexity deferred to the integration). [EFFORT_PROFILE]: Quick Win
Resolves #13760. Refs #13624, #13755.
Summary
Sub of #13624 — the pure hung-heavy-lease watchdog decision. The exclusive-heavy-maintenance lease releases on pid-death + TTL-expiry (
isLeaseStale), but NEITHER catches a holder that's ALIVE + within-TTL yet HUNG (sustained ~0% cpu — e.g. blocked on a wedged embedder). A hung holder monopolizes the lease until the full TTL, starving all other maintenance (the orchestrator DRAIN). Empirical: the 2026-06-21 01:17 incident — github-sync hung ~59min @0% cpu holding the lease idle.Distinct from the #13358/#13755 fall-through-skip regression (a fast 148ms no-op, not a hang); this is the HANG case — defense-in-depth so a hung embedder/task can never re-freeze the OS for a full TTL window.
Deltas
ai/daemons/orchestrator/services/leaseWatchdog.mjs(new): pureisHungLeaseHolder({cpuPercentSamples, idleThresholdPct=1, minConsecutiveIdle=3}) → boolean— true iff the last N cpu samples are ALL ≤ threshold (a sustained-idle hang; a single idle sample between work bursts is normal). Total + never-throws; fail-SAFE (non-array / too-few / non-finite → false — never force-release on bad/insufficient data).Out of scope (the integration)
The periodic ps-sampling of the active lease holder's pid + the force-release on a true verdict (the orchestrator lease-monitor loop) — to coordinate with @neo-opus-grace (orchestrator owner). Same pure-only carve as the RLAIF cores (#13724 / #13727). Authored as a new file (not in
HeavyMaintenanceLeaseService) to avoid colliding with Grace's live #13755 edit + because cpu-hang detection is a distinct concern from the basic pid/TTL stale-check.Test Evidence
Evidence: L2 — 6 unit tests green (
npm run test-unit -- leaseWatchdog.spec.mjs): sustained-idle → true; active-in-window → false; too-few → false; threshold-boundary; custom-window; fail-safe-on-malformed.check-jsdoc-typesclean;check-block-alignmentclean.Post-Merge Validation
isHungLeaseHolderis importable; the orchestrator lease-monitor can sample the active lease holder's cpu% over time + force-release when it returns true — releasing a hung holder in minutes instead of the full TTL.