Prevent back-to-back REM cycles after cadence overflow

Context

Operator surfaced live orchestrator logs on 2026-06-01 showing heavy-maintenance work never recovering after REM sleep graph extraction became active:

[2026-06-01T04:56:10.586Z] Deferring REM sleep graph extraction; heavy maintenance task primary checkout dev sync is active (periodic-dream:3600000).
[2026-06-01T04:57:37.145Z] Deferring agent OS backup; heavy maintenance task REM sleep graph extraction is active (periodic-sweep:86400000).
[2026-06-01T05:06:01.870Z] Deferring session summarization; heavy maintenance task REM sleep graph extraction is active (periodic-sweep:600000).
[2026-06-01T05:06:11.141Z] Deferring primary checkout dev sync; heavy maintenance task REM sleep graph extraction is active (periodic-sweep:600000).

Live V-B-A against the operator runtime found the heavy-maintenance lease in /Users/Shared/github/neomjs/neo/.neo-ai-data/orchestrator-daemon/heavy-maintenance-lease.json:

{
  "owner": "dream",
  "reason": "periodic-dream:3600000",
  "pid": 99816,
  "acquiredAt": "2026-06-01T04:56:13.701Z",
  "staleAfterMs": 21600000,
  "expiresAt": "2026-06-01T10:56:13.701Z"
}

The PID was alive. This is not simply a dead PID stale-lock case.

The Problem

The current scheduler can launch REM cycles back-to-back when a REM cycle exceeds its own cadence. The latest completed REM run on the operator host was:

{
  "runId": "rem-e47135e8-73dd-4456-8210-0ae3cd59bbad",
  "reason": "periodic-dream:3600000",
  "startedAt": "2026-06-01T03:28:18.681Z",
  "completedAt": "2026-06-01T04:56:00.800Z",
  "wallClockMs": 5262119,
  "configuredCadenceMs": 3600000,
  "cycleOverflowSignal": true,
  "cycleOverflowRatio": 1.4616997222222223,
  "outcome": "completed"
}

Because CadenceEngine.shouldRunIntervalTask() compares now - lastRunAt to the interval, and TaskStateService.markCompleted() does not advance lastRunAt, an 87.7-minute REM run on a 60-minute cadence is immediately due again after completion. The overflow warning from #12088 exists, but it is telemetry-only; no scheduling backoff consumes it. Result: REM can monopolize the shared heavy-maintenance lane and repeatedly defer session summaries, KB sync, primary dev sync, backup, and Golden Path dependency work.

The Architectural Reality

Relevant surfaces verified during diagnosis:

• ai/daemons/orchestrator/services/CadenceEngine.mjs: shouldRunIntervalTask({now, lastRunAt, intervalMs}) uses start-time cadence only.
• ai/daemons/orchestrator/services/TaskStateService.mjs: markStarted() updates lastRunAt; markCompleted() clears running and sets lastSuccessAt, but does not update a next-run anchor.
• ai/daemons/orchestrator/Orchestrator.mjs: dream scheduling passes lastRunAt : this.taskStateService.getTaskState('dream')?.lastRunAt and intervalMs: AiConfig.orchestrator.intervals.dreamMs.
• ai/daemons/orchestrator/services/DreamService.mjs: executeRemCycle() writes cycleOverflowSignal and logs a WARN, but the signal is not fed back into scheduling.
• ai/daemons/orchestrator/services/MaintenanceBackpressureService.mjs: the lease is released on promise settlement; the problem here is not proven to be release failure, but immediate reacquisition after cadence overflow.

This extends the exact operator framing captured in #12088: "if we trigger the process / task inside the orchestrator every hour => if it e.g. was 10m before and now magically grew to 90m (>the 60m repeat window) => trouble." #12088 built the observability; this ticket is the remediation layer.

The Fix

Implement a scheduling guard for dream so cadence overflow cannot create continuous REM occupation of the heavy-maintenance lane. Candidate shapes to evaluate during intake:

1. Anchor the next dream due time to completion time rather than start time for in-process long-running maintenance tasks.
2. Add explicit overflow backoff: when the last REM run has cycleOverflowSignal: true, require a recovery gap before another periodic REM run.
3. Use recent REM state (get_rem_pipeline_state / JSONL store) to compute an adaptive cooldown, while keeping manual REM runs possible.

The implementation should be narrow and test-backed. It should not weaken the heavy-maintenance mutex itself.

Contract Ledger Matrix

Decision Record Impact

Aligned with ADR 0009 (cross-daemon heavy-maintenance lease inheritance). This ticket should not weaken the lease contract; it changes scheduler fairness/backoff around the dream lane.

Acceptance Criteria

• Periodic dream scheduling cannot immediately start a new REM cycle solely because the previous run exceeded the configured cadence while measuring from start time.
• A REM run with cycleOverflowSignal: true produces a concrete recovery behavior: completion-time cooldown, overflow backoff, or equivalent fairness guard.
• Other heavy-maintenance tasks (summary, kbSync, primary-dev-sync, backup) get an opportunity to run after an overflowed REM cycle completes.
• Manual/operator-triggered REM remains possible and does not bypass lease safety.
• Unit coverage simulates an 87-minute REM run on a 60-minute cadence and verifies the next periodic dream poll defers rather than reacquires immediately.
• Unit coverage verifies non-overflowed cycles preserve normal cadence behavior.
• Documentation or runbook text explains how cycleOverflowSignal affects scheduling and operator interpretation.

Out of Scope

• Changing local model/provider selection, LMS preload behavior, or embedding/chat model defaults (#12264 and related provider tickets own that).
• Removing the heavy-maintenance lease file by hand as the primary fix.
• Changing REM extraction quality or tri-vector prompt behavior.
• Solving REM JSONL retention/prune (#12123 owns that).

Avoided Traps

• Do not classify this as a dead PID stale lock just because other tasks are deferred. The lease holder PID is alive; the live evidence points to repeated long-running dream cycles.
• Do not stop at more telemetry. #12088 already made overflow visible; the current failure is that scheduling ignores it.
• Do not disable REM entirely as the architectural fix. The goal is fairness/recovery between heavy maintenance lanes.
• Do not rely on wall-clock feelings or fan noise. Use lease payload, task state, REM JSONL, and orchestrator logs as falsifiers.

Related

• #12088 — REM run/stage state model + cadence-overflow detection (closed; observability layer)
• #12068 / #12065 — REM observability epic/sub context
• #12123 — REM JSONL retention/prune (related storage hygiene, not scheduler fairness)
• #12264 — LMS preload regression (related local-model pressure, not this scheduler defect)
• ADR 0009 — cross-daemon heavy-maintenance lease inheritance

Origin Session ID: 019e7f45-ad55-75e0-bfff-a21c2385df00

Handoff Retrieval Hints: query_raw_memories("periodic-sweep heavy maintenance lease stale lock orchestrator REM sleep graph extraction"); ask_knowledge_base("heavy maintenance lease periodic dream REM cadence overflow scheduler"); live files .neo-ai-data/orchestrator-daemon/heavy-maintenance-lease.json, .neo-ai-data/orchestrator-daemon/orchestrator-state.json, .neo-ai-data/rem-runs/rem-e47135e8-73dd-4456-8210-0ae3cd59bbad.jsonl.