LearnNewsExamplesServices
Frontmatter
id16267
titleA wake route published after the receiver boots is permanently deaf
stateClosed
labels
bugaiarchitecture
assigneesneo-opus-ada
createdAtAug 1, 2026, 4:24 PM
updatedAtAug 1, 2026, 5:39 PM
githubUrlhttps://github.com/neomjs/neo/issues/16267
authorneo-opus-ada
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtAug 1, 2026, 5:39 PM

A wake route published after the receiver boots is permanently deaf

Closed Backlog/active-chunk-11 bugaiarchitecture
neo-opus-ada
neo-opus-ada commented on Aug 1, 2026, 4:24 PM

Context

On 2026-08-01 three of four seats in the wake cohort had no delivery, with one symptom and three different causes. One of them is structural and is this ticket.

Measured on the live system rather than reasoned about. A forged-signature POST /wake per subscription id returns 401 invalid-signature if the running receiver holds that route and 404 unknown-subscription if it does not — a read-only probe that answers what the process holds, not what the file says:

ROUTE ON DISK                     RUNNING RECEIVER (pid 65803, booted 13:04:10Z)
36342c42  @neo-opus-ada           401  LOADED
90021c89  @neo-kimi-phoebe        401  LOADED
73a0e4ed  @neo-gpt-emmy           404  INVISIBLE   ← published 13:13:07Z
3a6ada00  @neo-gpt                404  INVISIBLE   ← published ~13:33Z

Both invisible routes were correctly generated, correctly signed, present on disk at mode 0600, and validated by the receiver's own loader. They were simply published after the process started.

The Problem

The receiver reads its manifest exactly once, at boot. loadWakeReceiverManifest (ai/daemons/wake/receiver.mjs:48) is called by startWakeReceiver and the resulting object is handed to createWakeReceiver({manifest, …}) at :147. There is no SIGHUP handler, no fs.watch, and no re-read anywhere in the file.

The generator is built for the opposite lifecycle. buildWakeReceiverManifest composes additively — it seeds from existingRoutes so one seat cannot unprovision a peer, and each seat publishes its own route independently (#16233). That design is correct and deliberate. It assumes routes arrive over time.

So the two halves assume different lifecycles: publishing is not provisioning. A seat that runs the documented command, sees published 1 route(s), and validates its manifest has done everything right and is still deaf.

And the failure is terminal on the first wake, not gradual. A route the receiver does not hold returns 404. WebhookDeliveryService treats any 4xx as a client error and marks the subscription degraded immediately, with no retry (ai/services/memory-core/WebhookDeliveryService.mjs:133-135). Observed: Emmy published at 13:13:07Z and her row degraded at 13:13:10Z — three seconds later, on her first wake.

The operational trap this creates. The remedy is a restart, which is also the action an operator is most likely to be told to avoid during an incident — during this one, "publish missing Shape-B routes; DO NOT restart :3199" was live guidance. Both halves were individually correct and jointly unsatisfiable: a published route only goes live on reload, and the only reload path is a restart. The cohort stalled until that was noticed.

The Architectural Reality

Surface Role
ai/daemons/wake/receiver.mjs:48 loadWakeReceiverManifest — the single read
ai/daemons/wake/receiver.mjs:147 createWakeReceiver({manifest, …}) — takes a loaded object, holds it for process lifetime
ai/daemons/wake/receiver.mjs:208 route lookup → 404 unknown-subscription when absent
ai/daemons/wake/buildReceiverManifest.mjs additive composition; assumes routes accrue over time
ai/services/memory-core/WebhookDeliveryService.mjs:133-135 4xx → degrade immediately, no retry

The receiver is otherwise correct to be conservative: it validates the manifest at load, refuses anything the loader rejects, and never re-reads a file that could change under it mid-dispatch. The gap is that nothing re-reads it, ever.

The Fix

Not prescribing the mechanism — the options differ in blast radius and the choice belongs with whoever owns the receiver's runtime posture.

  1. Reload on a signal. SIGHUP re-reads and revalidates, keeping the same fail-closed semantics as boot: a manifest the loader rejects leaves the current routes in place rather than emptying them. Cheapest, and composes with a launchd-managed process.
  2. Reload on change. Watch the manifest path, debounce, revalidate before swapping. More moving parts, and needs care so a partially-written file is never adopted — the generator already publishes via an exclusive staging file plus rename, so an atomic-rename watch is viable.
  3. Make the generator signal the receiver. Couples the two components that are currently independent; mentioned for completeness, and it is the option I would argue against.

Whichever lands, a 404 for a route the operator believes is published should not be silently terminal. Either the degrade should not fire on a 404 that may reflect receiver staleness, or the reason must reach the record so the seat owner can see unknown-subscription rather than nothing. (The reason-plumbing half is #16259 / PR #16264.)

Contract Ledger

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
Receiver route table the manifest on disk reflects the file without a process restart invalid manifest ⇒ keep current routes, never empty runbook a route published after boot answers 401, not 404
Reload trigger this ticket one named, documented mechanism absent trigger ⇒ current boot-only behaviour, documented as such runbook the runbook's publish step names how the route goes live
404-driven degrade WebhookDeliveryService:133-135 a 404 does not silently terminalize a valid route degrade retained for genuine 4xx this ticket a stale-receiver 404 is distinguishable from a rejected route

Decision Record impact

none. This restores the lifecycle the generator was designed for; it does not change the manifest contract, the signing model, or the routing shape.

Acceptance Criteria

  • A route published while the receiver is running becomes deliverable without restarting the process, asserted with the 401/404 probe against a live receiver — not by reading the manifest back.
  • A manifest that fails validation during reload leaves the previously-loaded routes intact; asserted by reloading a deliberately-invalid manifest and confirming an existing route still answers 401.
  • A partially-written manifest is never adopted; asserted against the staging/rename path the generator already uses.
  • A seat publishing a route and receiving a wake immediately afterwards is not degraded by a stale receiver, or the resulting record names unknown-subscription as the cause.
  • The runbook's publish step states how a published route becomes live, so the two documents describe one lifecycle.

Out of Scope

  • The generator's additive composition (#16233) — correct as designed; the mismatch is receiver-side.
  • The adapter's discarded failure reason — #16259 / PR #16264.
  • The degrade/resume semantics themselves — #16246 / #16253, both merged.
  • Wake delivery, signing, and the GUI tuple contract; all verified working (#16233, issuecomment-5151143667).

Avoided Traps

  • Reloading without revalidation. A malformed or truncated manifest must not empty a working route table mid-incident; that converts a stale receiver into a dead one.
  • Watching the file naively. The publish path writes a staging file and renames; a watcher that reads on every event will occasionally read a file that is not the finished one.
  • Treating this as a documentation fix. The runbook could say "restart after publishing" and the trap would remain, because the restart is exactly what incident guidance tends to forbid.
  • Blaming the reviewer or the publisher. Every route in the observed incident was correctly generated and correctly published. The seats did nothing wrong; the lifecycle assumption did.

Related

  • #16233 — the generator whose additive design this must not disturb
  • #16259 / PR #16264 — the discarded-reason half; a 404 here currently reaches the record as no reason at all
  • #16246, #16253 — degrade and resume, both merged
  • #16258 — routes going silent after a Memory Core restart; adjacent, different component

Live latest-open sweep: checked the latest 14 open issues at 2026-08-01T14:23Z plus targeted searches for receiver/manifest reload, publishing-vs-provisioning lifecycle, and wake-route 404 degrade; only the #16167 umbrella matched and it does not name this. A2A in-flight claim scan run against the same window; no overlapping claim.

Origin Session ID: 56105163-6e66-44b6-8c6f-9e81bc1be08c

Retrieval Hint: wake receiver manifest boot snapshot no reload SIGHUP published route 404 unknown-subscription immediate degrade publishing is not provisioning

tobiu referenced in commit 88e6ebd - "feat(wake): reload the receiver manifest without a restart (#16267) (#16271) on Aug 1, 2026, 5:39 PM
tobiu closed this issue on Aug 1, 2026, 5:39 PM