LearnNewsExamplesServices
Frontmatter
id11872
titleFix Codex nightshift wake driver gap in Orchestrator heartbeat
stateClosed
labels
bugaiarchitecturemodel-experience
assigneesneo-gpt
createdAtMay 24, 2026, 3:18 AM
updatedAtJun 7, 2026, 7:15 PM
githubUrlhttps://github.com/neomjs/neo/issues/11872
authorneo-gpt
commentsCount8
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtMay 25, 2026, 6:43 PM

Fix Codex nightshift wake driver gap in Orchestrator heartbeat

Closed v13.0.0/archive-v13-0-0-chunk-13 bugaiarchitecturemodel-experience
neo-gpt
neo-gpt commented on May 24, 2026, 3:18 AM

Summary

Codex nightshift wakeup is currently not durable through the Orchestrator heartbeat path. I re-enabled the local Codex heartbeat automation as an immediate workaround, but that is a harness-side heartbeat, not the Orchestrator-owned swarm heartbeat contract the nightshift mode expects.

This is bleeding-edge Agent OS substrate that has not shipped externally. No migration or backwards-compatibility work is needed here; the correct fix should simplify the current shape rather than preserving unreleased drift.

Problem

Operator report on 2026-05-24: Claude gets watchdog wakeup from the Orchestrator, but Codex does not. Current evidence shows the Codex SENT_TO_ME bridge route exists, yet the broader nightshift driver path is not reliable for @neo-gpt:

  • manage_wake_subscription(list) shows active @neo-gpt SENT_TO_ME bridge route WAKE_SUB:dc10c816-ae36-48eb-8074-a36625581d52 targeting Codex.
  • Memory Core healthcheck reports wake gate enabled, but daemonRunning: false; lastPulseAt: 2026-05-23T20:53:11.846Z.
  • The local Codex automation nightshift-lifecycle-driver was present but PAUSED; I re-enabled it to ACTIVE at 15-minute cadence as a temporary operator-facing workaround.
  • SwarmHeartbeatService initializes a single identity (identity || NEO_AGENT_IDENTITY || @neo-gemini-pro) and falls back to tmux injection for generic heartbeat prompts. Codex Desktop does not receive that tmux path in this harness.
  • resumeHarness.mjs does contain an @neo-gpt -> codex-desktop -> codex-app-server mapping, but that path only matters when the heartbeat reaches sunset/recovery dispatch. It does not make the routine nightshift lifecycle-driver wake durable by itself.
  • #11797 validated a controlled @neo-gpt SENT_TO_ME bridge delivery after #11804, but that does not prove autonomous Codex nightshift driving when the Orchestrator heartbeat is stale or when the generic pulse path is tmux-only.

Source Of Authority

  • Operator directive, 2026-05-24: enable watchdog wakeup for @neo-gpt; Claude gets it from Orchestrator, Codex does not; treat as bug.
  • learn/agentos/wake-substrate/NightShiftLeasedDriver.md: nightshift driver lease is lane-scoped and distinct from #10671 wake delivery.
  • learn/agentos/wake-substrate/PersistentProcessManagement.md: since #11766 the swarm heartbeat is an Orchestrator lane, not a separate daemon.
  • ai/daemons/orchestrator/services/SwarmHeartbeatService.mjs: current single-identity pulse + tmux fallback behavior.
  • ai/scripts/lifecycle/resumeHarness.mjs: @neo-gpt maps to codex-desktop / codex-app-server for recovery dispatch.
  • #11797 / #11804: identity normalization and controlled bridge delivery validation, but not the complete Codex nightshift-driver contract.

Contract Ledger

Field Value
Target @neo-gpt Codex Desktop nightshift watchdog / lifecycle driver
Current temporary state Codex heartbeat automation nightshift-lifecycle-driver re-enabled, ACTIVE, 15-minute cadence
Durable owner Orchestrator swarm-heartbeat lane, not a per-thread manual workaround
Transport Existing wake substrate only: Memory Core A2A + bridge-daemon / resumeHarness; do not add a second heartbeat primitive
Scope Unreleased v13 Agent OS substrate; no migration/backwards-compatibility requirement
Related #10396, #10671, #11766, #11797, #11804

Acceptance Criteria

  1. Reproduce or falsify the current split with live evidence: @neo-gpt has an active SENT_TO_ME bridge route, while the Orchestrator heartbeat path is not durably waking Codex nightshift work without the local Codex automation.
  2. Make the Orchestrator-owned heartbeat contract explicit for @neo-gpt: either the heartbeat lane drives Codex nightshift lifecycle work directly, or the supported fallback is formally surfaced as an operator-visible health/config state.
  3. Do not create a new heartbeat primitive. Reuse the existing #10671 / #11766 wake substrate (SwarmHeartbeatService, idleOutNudge, trioWakeCooldown, resumeHarness, bridge-daemon).
  4. Add focused unit coverage for the @neo-gpt / Codex path. At minimum cover identity selection/routing and the no-tmux Codex Desktop case that currently lets generic pulse injection no-op.
  5. Update the wake substrate runbook (PersistentProcessManagement.md and/or NightShiftLeasedDriver.md) so nightshift readiness includes Codex route validation, not just generic heartbeat liveness.
  6. Post-fix validation proves whether the temporary Codex automation can be disabled. If it must stay, its status must be observable as part of nightshift readiness and not silently drift to PAUSED.

Out Of Scope

  • Rewriting the wake substrate from scratch.
  • Adding a second scheduler/heartbeat daemon.
  • Solving all Codex app-server live-host proof work unless it is strictly required for this route.
  • Backwards compatibility with unreleased intermediate shapes.

Avoided Traps

  • Do not treat #11797 as complete proof of this contract. It proves controlled @neo-gpt message delivery, not autonomous nightshift driver coverage.
  • Do not infer heartbeat health from log silence alone; use Memory Core healthcheck wake fields and Orchestrator task state.
  • Do not rely on tmux injection as the Codex Desktop path.

Verification Notes

Already checked before filing:

  • manage_wake_subscription({action: 'list'}) -> active @neo-gpt bridge route WAKE_SUB:dc10c816-ae36-48eb-8074-a36625581d52.
  • memory_core.healthcheck() -> wake gate enabled, daemonRunning: false, stale lastPulseAt.
  • Source read: SwarmHeartbeatService.mjs, Orchestrator.mjs, resumeHarness.mjs, NightShiftLeasedDriver.md, PersistentProcessManagement.md.
  • Duplicate/context sweep: #11797, #10396, local issue/pr resources for swarm-heartbeat and Codex wake routing.
tobiu referenced in commit de9d3fe - "fix(ai): route heartbeat through active wake subscriptions (#11872) (#11875) on May 24, 2026, 4:21 AM
tobiu referenced in commit 329208b - "fix(orchestrator): align heartbeat liveness path (#11872) (#11958) on May 25, 2026, 9:51 AM