LearnNewsExamplesServices
Frontmatter
id16253
titleRestoring a degraded wake route needs a restart: nothing invokes clearDegraded
stateClosed
labels
bugai
assigneesneo-opus-grace
createdAtAug 1, 2026, 12:56 PM
updatedAtAug 1, 2026, 1:27 PM
githubUrlhttps://github.com/neomjs/neo/issues/16253
authorneo-kimi-iris
commentsCount0
parentIssue16246
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 1, 2026, 1:27 PM

Restoring a degraded wake route needs a restart: nothing invokes clearDegraded

neo-kimi-iris
neo-kimi-iris commented on Aug 1, 2026, 12:56 PM

Context

Filed from the Approve+Follow-Up review of PR #16251 (pullrequestreview-4834513251, reviewId PRR_kwDODSospM8AAAABICjRYw), which names this as the durable follow-up — and from a live case the same morning: @neo-opus-ada's seat hit the full degrade chain (four failures → status: degraded persisted), and restoring it required working around the very gap this ticket covers.

PR #16249/#16251 gave a dead wake route durable truth: _markDegraded writes properties.status: 'degraded' onto the node (readable by checkSunsetted and the heartbeat exclusion), delivery attempts are bounded by a dual skip (properties.status === 'degraded' || degradedSubscriptions.has(id), WebhookDeliveryService.mjs:86), and clearDegraded exists as the named resumption path (WebhookDeliveryService.mjs:223).

The Problem

The resumption path exists, but nothing can reach it.

clearDegraded(subscriptionId) clears only the in-memory markers (degradedSubscriptions, consecutiveFailures) — by design, per its JSDoc: the restorer must also move the persisted properties.status off 'degraded' "in the same operation". Two consequences, both verified at source:

  1. No production caller exists. git grep clearDegraded at the merge head hits only the method, its JSDoc link, and its spec. An operator restore today means: create a NEW subscription (fresh id), or hand-edit status via a graph path AND get lucky with timing, or restart the mc-server process (which empties the in-memory set but leaves the persisted status degraded — so the route stays skipped anyway, since the flush re-reads properties.status fresh).
  2. The single-phase call is a footgun. A restorer calling only clearDegraded resumes nothing: the next flush's fresh properties.status still reads degraded, so the route stays skipped. The correct usage is two-phase (status move, then clearDegraded) — modeled by the PR's spec — but nothing in the tool surface performs or documents that sequence where an operator would find it.

The Architectural Reality

  • WebhookDeliveryService.mjs:86 — the skip reads two degraded truths: the flush's fresh properties.status AND the in-memory degradedSubscriptions set. Both must be clear for delivery to resume.
  • WebhookDeliveryService.mjs:223-226clearDegraded clears the in-memory half only. Its JSDoc assigns the persisted-status move to "whoever restores", but no caller implements the assignment.
  • manage_wake_subscription (MCP tool) is the operator's natural restore surface — it owns subscribe/unsubscribe/update — and has no path that touches either truth. The restore contract belongs there: one action that moves properties.status off 'degraded' AND invokes clearDegraded, atomically.

The Fix

Wire the resumption into the operator surface:

  1. manage_wake_subscription gains a restore path (preferred: the existing update action re-activating a route, or an explicit clear/resume action) that, for a degraded subscription id, performs BOTH halves in one operation: upsertNode({id, properties: {status: 'active'}}) and WebhookDeliveryService.clearDegraded(id). The JSDoc's two-phase contract becomes the tool's single atomic step.
  2. Tighten clearDegraded's JSDoc first line (from the review's inline nit): "clears the in-memory markers; the restorer must also move properties.status off 'degraded'" — so the method stops reading as self-sufficient.
  3. Specs: (a) the wired path resumes delivery end-to-end (degrade → restore action → next flush delivers, both truths clear); (b) the footgun case pinned — clearDegraded alone does NOT resume (documents why the two-phase is mandatory); (c) manage_wake_subscription list after restore shows status off 'degraded'.

Alternative considered and rejected: making clearDegraded itself move the persisted status. It would violate the PR's deliberate design (the service's delivery path should not silently re-activate routes — re-activation is an operator act, and the review already ratified the two-phase contract). The wiring belongs at the operator surface, not inside the delivery service.

Acceptance Criteria

  • One operator-reachable action (MCP tool or named lifecycle path) restores a degraded route: persisted status moved off 'degraded' AND in-memory markers cleared, in one atomic step.
  • End-to-end spec: degrade → restore → next flush delivers (both skip truths verified clear).
  • Footgun spec: clearDegraded alone does not resume delivery (the persisted status still skips) — the two-phase requirement is pinned, not tribal.
  • clearDegraded JSDoc first line names the two-phase contract explicitly.
  • No new silent re-activation path: restore remains an explicit operator act with a log line at INFO.

Out of Scope

  • Any change to the degrade mechanics, threshold, skip dual-read, or the RLS model (all shipped and ratified by PR #16251).
  • A boundedRetryGate adoption for per-subscription delivery state — a family-level design question noted in the PR #16251 review closing, tracked in the #16222-family thread.
  • Receiver-side provisioning (#16233, shipped) and FM-hub wake delivery for adapter-less harnesses (D#16247).

Related

Source: PR #16251 (A+FU review pullrequestreview-4834513251) · #16246 (the degrade mechanics) · #16233 / PR #16249 (receiver side) · live case: @neo-opus-ada's 2026-08-01 degrade/restore chain (A2A MESSAGE:9ca5f0d3).

Live latest-open sweep: checked latest 20 open issues at 2026-08-01T11:00Z; no equivalent found. A2A in-flight sweep (last 30 messages, all read-states): no competing claim on this scope. Local exact sweep (clearDegraded, restore/resumption+wake): no open ticket. KB semantic sweep: unavailable (collection mid-rebuild, count 0) — substituted live GitHub + grep.

Origin Session ID: 05b5fdc9-1f2b-4b45-a2c9-4b64ed5f15cd

Retrieval Hint: query_raw_memories("clearDegraded resumption wiring manage_wake_subscription restore degraded wake route two-phase")

tobiu referenced in commit ad76ede - "feat(wake): an operator-reachable resume for a degraded route (#16253) (#16255) on Aug 1, 2026, 1:27 PM
tobiu closed this issue on Aug 1, 2026, 1:27 PM