Context
Found 2026-08-10 while diagnosing a CPU-only cloud deployment whose orchestrator showed restartCount: 12 with exitCode: 0, oomKilled: false, error: null on every exit. Those readings are what a graceful shutdown looks like, so the restarts read as "somebody stopped it twelve times" rather than "it crashed twelve times". They are indistinguishable at the container level, and the reason is in our source.
Plan-Authority: INDEPENDENT — runtime defect, no governing epic. Relevant to #16706's external-plane diagnosability but not scoped by it.
The Problem
All four long-lived daemons route their uncaughtException handler into the same cleanup() that SIGTERM uses, and that cleanup() calls bare process.exit(). Node's process.exit() with no argument exits 0. Nothing in these paths sets process.exitCode.
| daemon |
handler |
cleanup() exit |
ai/daemons/orchestrator/daemon.mjs |
:203 |
:197 process.exit() → 0 |
ai/daemons/embed/daemon.mjs |
:279 |
:273 process.exit() → 0 |
ai/daemons/message/daemon.mjs |
:212 |
:206 process.exit() → 0 |
ai/daemons/wake/daemon.mjs |
:518 |
:512 process.exit() → 0 |
Two of them carry the comment // Calls process.exit() automatically, which documents that it exits and not with what.
The consequence is that a crash is reported as success to everything downstream of the process boundary:
- A container runtime cannot distinguish a crash from an operator stop.
restart: unless-stopped restarts either way, so the daemon does come back — the loss is not availability, it is attribution.
- Any alerting, healing, or supervision keyed on a non-zero exit never fires. On the deployment that surfaced this, the self-heal ledger reads
total: 0 with no events ever recorded, beside a container that had restarted twelve times.
docker inspect gives the operator exitCode: 0 and error: null, so the natural next step — "nothing crashed, check who restarted it" — is the wrong investigation, pointed away from the stack trace that was written seconds earlier.
Availability is not the defect; observability is. The restart policy masks it, which is exactly why it has survived.
The Architectural Reality
- The three non-orchestrator daemons share a near-identical
cleanup() (idempotent via cleanedUp, unlink-own-pidfile, exit). The orchestrator's differs slightly (Orchestrator.stop(), removePidFile()) but ends the same way.
SIGINT/SIGTERM → cleanup() is correct at exit 0. A signal-initiated stop is success. Only the uncaughtException arm is wrong, so the repair must not flatten the two into one code.
process.on('exit', removePidFile) in the orchestrator means pid-file cleanup already survives an exit-code change.
- Non-zero exits do exist elsewhere in these files (
embed:52/254/300, message:33/189/227, orchestrator:455, hostEdge:74), so the codebase's convention is that a failure exits non-zero — these four crash paths are the deviation, not the norm.
The Fix
- The
uncaughtException arm exits non-zero. Pass an explicit failure code through cleanup(code) (defaulting to 0) so the signal path keeps exit 0 and the crash path does not. The same for any unhandledRejection handler that shares the path.
- Do it in all four, and prove the shared shape rather than fixing one and copying by eye. Three of the four
cleanup() bodies are already textually identical; a repair that lands in one and not the others reproduces this ticket in a year.
- Keep the stack log. It is already written before
cleanup() and is the only surviving evidence today; nothing about the exit-code change should reorder it.
Contract Ledger
| Target Surface |
Source of Authority |
Proposed Behavior |
Fallback |
Docs |
Evidence |
| daemon crash exit |
this ticket |
uncaughtException exits non-zero |
unchanged pid-file cleanup and stack log |
JSDoc on each cleanup() stating which code each caller yields |
unit: a handler-invoked cleanup yields non-zero |
| daemon signal exit |
existing behavior |
SIGINT/SIGTERM continue to exit 0 |
unchanged |
same JSDoc |
unit: signal path still yields 0 — the non-vacuity arm |
Acceptance Criteria
Out of Scope
- Why any particular daemon threw. This ticket makes crashes visible; it diagnoses none of them.
- Restart policy, backoff, or supervision changes.
Context
Found 2026-08-10 while diagnosing a CPU-only cloud deployment whose orchestrator showed
restartCount: 12withexitCode: 0,oomKilled: false,error: nullon every exit. Those readings are what a graceful shutdown looks like, so the restarts read as "somebody stopped it twelve times" rather than "it crashed twelve times". They are indistinguishable at the container level, and the reason is in our source.Plan-Authority: INDEPENDENT — runtime defect, no governing epic. Relevant to #16706's external-plane diagnosability but not scoped by it.
The Problem
All four long-lived daemons route their
uncaughtExceptionhandler into the samecleanup()that SIGTERM uses, and thatcleanup()calls bareprocess.exit(). Node'sprocess.exit()with no argument exits 0. Nothing in these paths setsprocess.exitCode.cleanup()exitai/daemons/orchestrator/daemon.mjs:203:197process.exit()→ 0ai/daemons/embed/daemon.mjs:279:273process.exit()→ 0ai/daemons/message/daemon.mjs:212:206process.exit()→ 0ai/daemons/wake/daemon.mjs:518:512process.exit()→ 0Two of them carry the comment
// Calls process.exit() automatically, which documents that it exits and not with what.The consequence is that a crash is reported as success to everything downstream of the process boundary:
restart: unless-stoppedrestarts either way, so the daemon does come back — the loss is not availability, it is attribution.total: 0with no events ever recorded, beside a container that had restarted twelve times.docker inspectgives the operatorexitCode: 0anderror: null, so the natural next step — "nothing crashed, check who restarted it" — is the wrong investigation, pointed away from the stack trace that was written seconds earlier.Availability is not the defect; observability is. The restart policy masks it, which is exactly why it has survived.
The Architectural Reality
cleanup()(idempotent viacleanedUp, unlink-own-pidfile, exit). The orchestrator's differs slightly (Orchestrator.stop(),removePidFile()) but ends the same way.SIGINT/SIGTERM→cleanup()is correct at exit 0. A signal-initiated stop is success. Only theuncaughtExceptionarm is wrong, so the repair must not flatten the two into one code.process.on('exit', removePidFile)in the orchestrator means pid-file cleanup already survives an exit-code change.embed:52/254/300,message:33/189/227,orchestrator:455,hostEdge:74), so the codebase's convention is that a failure exits non-zero — these four crash paths are the deviation, not the norm.The Fix
uncaughtExceptionarm exits non-zero. Pass an explicit failure code throughcleanup(code)(defaulting to0) so the signal path keeps exit 0 and the crash path does not. The same for anyunhandledRejectionhandler that shares the path.cleanup()bodies are already textually identical; a repair that lands in one and not the others reproduces this ticket in a year.cleanup()and is the only surviving evidence today; nothing about the exit-code change should reorder it.Contract Ledger
uncaughtExceptionexits non-zerocleanup()stating which code each caller yieldsSIGINT/SIGTERMcontinue to exit0Acceptance Criteria
uncaughtExceptionhandler runs, proven per-daemon. A spec that covers one daemon and asserts the others by inspection does not satisfy this.0— asserted in the same spec. Without this arm the change could set every exit non-zero and still go green, which would make every graceful stop look like a crash and is strictly worse than today.// Calls process.exit() automaticallycomments state the resulting code.Out of Scope