LearnNewsExamplesServices
Frontmatter
id16352
titleThe wake receiver should follow the manifest, not wait to be told
stateClosed
labels
enhancementai
assigneesneo-opus-grace
createdAtAug 2, 2026, 3:16 PM
updatedAtAug 2, 2026, 6:49 PM
githubUrlhttps://github.com/neomjs/neo/issues/16352
authorneo-opus-grace
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 2, 2026, 6:49 PM

The wake receiver should follow the manifest, not wait to be told

Closed Backlog/active-chunk-12 enhancementai
neo-opus-grace
neo-opus-grace commented on Aug 2, 2026, 3:16 PM

Context

Operator direction 2026-08-02: "restarting the wake service manually on EVERY wake subscription change is also more than brittle. it should always USE the latest state."

Successor to #16267 (CLOSED/COMPLETED 2026-08-01), which diagnosed "a wake route published after the receiver boots is permanently deaf" and shipped the SIGHUP handler. That closed the "impossible" half. This ticket is the "manual" half it deliberately left: a reload that must be asked for is still a step someone has to remember, and today it cost an outage.

Live latest-open sweep 2026-08-02T13:15:52Z. #16267 read in full before filing.

The Problem

ai/daemons/wake/receiver.mjs re-reads its manifest only on SIGHUP (:352). There is no fs.watch, no TTL, no per-request re-read. So every subscription change needs an out-of-band human or agent action, and the receiver's own banner advertises the signal as the way to do it:

[Wake Receiver] listening on http://127.0.0.1:3199/wake — SIGHUP reloads the manifest

Both failure modes fired today, hours apart.

1. Publish without reload is silent. I repaired my own route and published it additively (7 → 8 routes, manifest correct at 0600). The running receiver kept serving 7. Nothing on either side reported a discrepancy — the file said 8, the process said 7, and only an external observer comparing them could tell.

2. The documented reload killed the receiver. I then ran kill -HUP on it. The handler exists in dev; the running process had been up since 02:18 and predated it, so SIGHUP hit Node's default disposition and terminated the daemon — taking all eight routes down, including seven that were working. The operator restarted it by hand.

That second one is the sharper argument. The signal is only safe if you first establish that the running build is newer than the handler that makes it safe — a precondition nothing surfaces, on an action whose failure mode is everyone's outage rather than the caller's.

And the manual step compounds with drift. A hand-started receiver is exactly the kind that lags dev; a lagging receiver is exactly the one where the documented signal is fatal. The two defects are correlated, not independent.

The Architectural Reality

  • startWakeReceiver (receiver.mjs:315) loads the manifest once and hands the object to createWakeReceiver.
  • reload() (:337) re-reads, revalidates and swaps — and already fails safe: a load error leaves serving routes untouched, with the comment "an unreadable or malformed file must never empty a working route table — that turns a stale receiver into a dead one, mid-incident." That property is the hard part and it is done.
  • process.on('SIGHUP', …) (:352) is the only trigger.
  • The receiver has no supervisor: ~/Library/LaunchAgents/ holds only com.neomjs.middleware-rebuild.plist. It is started by hand, so nothing restarts it after a crash either.

The generator (buildReceiverManifest.mjs) already publishes atomically under withOutboxLock and validates through the receiver's own loader before writing, so a watcher would never observe a torn file.

The Fix

(Prescription — the trigger is the decision; reload() itself is already correct and must not be rewritten.)

  1. fs.watch on the manifest path, debounced, calling the existing reload(). Cheapest, and the publish path's atomic rename makes the event clean. Watch semantics differ across platforms and can miss or double-fire, so it needs a debounce and probably a periodic reconcile as a backstop.
  2. Re-read per request when the incoming subscriptionId is unknown, before answering 404. Precise and self-limiting — the miss is exactly the moment the stale snapshot matters — but it puts a filesystem read on a request path an attacker can drive, so it needs rate-limiting.
  3. Short TTL re-read, e.g. re-read if the manifest mtime changed and the last check is older than N seconds. Simplest to reason about, bounded work, at the cost of a small deaf window.

My read is 1 with 3 as the backstop — watch for latency, periodic reconcile so a missed event self-heals. But fs.watch's cross-platform behaviour is the real risk and the choice belongs in this ticket.

Whatever is chosen, SIGHUP stays. It is the documented escape hatch and #16267's delivered contract; this removes the need for it, not the ability.

Acceptance Criteria

  • A route published while the receiver is running becomes servable without any external signal or restart.
  • The existing fail-safe is preserved: an unreadable or malformed manifest leaves the serving route table untouched. A spec proves a corrupt write does not empty it.
  • A missed or duplicated filesystem event self-heals — the receiver converges on the file's contents without operator action. A spec covers a change that fires no watch event.
  • SIGHUP still reloads.
  • Route-table changes are logged with the resulting count, so "did it pick it up" is answerable from the log rather than by probing.
  • Post-merge: publish a route to a running receiver, send nothing, and observe delivery.

Out of Scope

  • Supervising the receiver process (auto-restart, launchd/plist). Real adjacent friction — it has no supervisor today — but a different deliverable; file separately if wanted.
  • The --initialize guard (#16344) and backup capture (#16348).
  • Arming seats at boot (#16310) — that is the subscription side; this is the receiver side.

Avoided Traps

  • Rewriting reload(). It is correct, including the fail-safe that is easy to get wrong. This ticket adds a trigger and touches nothing inside it.
  • Treating SIGHUP as the fix. It is #16267's delivered mechanism and it works — on a build that has it. Today's outage was not SIGHUP being wrong; it was a manual step requiring knowledge nothing surfaces.
  • Assuming a watch event always arrives. fs.watch misses events on some platforms and filesystems. A watcher with no reconcile trades a known manual step for an unknown silent one, which is worse.

Related

  • #16267 — CLOSED/COMPLETED; diagnosed boot-snapshot deafness and shipped SIGHUP. Direct predecessor.
  • #16310 — nothing arms a route at boot (subscription side of the same darkness).
  • #16323 — the arming verdict; reports the Memory-Core leg only, precisely because the receiver leg is invisible to it.

Origin Session ID: 713db0da-2239-44ea-ba5b-931be90d34fc

Retrieval Hint: query_raw_memories("wake receiver boot snapshot SIGHUP manual reload latest manifest state")

tobiu referenced in commit 7cbb189 - "feat(wake): the receiver follows its manifest instead of waiting to be told (#16352) (#16373) on Aug 2, 2026, 6:49 PM
tobiu closed this issue on Aug 2, 2026, 6:49 PM