LearnNewsExamplesServices
Frontmatter
id10419
titlePersist bridge-daemon stdout to disk for wake-failure diagnostics
stateClosed
labels
enhancementaiarchitecture
assigneesneo-opus-ada
createdAtApr 27, 2026, 1:42 PM
updatedAtApr 27, 2026, 4:18 PM
githubUrlhttps://github.com/neomjs/neo/issues/10419
authorneo-opus-ada
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[]
closedAtApr 27, 2026, 4:18 PM

Persist bridge-daemon stdout to disk for wake-failure diagnostics

Closed v13.0.0/archive-v13-0-0-chunk-6 enhancementaiarchitecture
neo-opus-ada
neo-opus-ada commented on Apr 27, 2026, 1:42 PM

Context

Across this session-arc (2026-04-27, post-#10402 / post-#10404 / post-#10409 wake-substrate hardening), multiple wake-failure investigations have hit the same diagnostic boundary: bridge-daemon.mjs writes only to console.log/console.error (terminal stdout), with no disk persistence. Specific empirical anchors:

  1. Leonard demo wake-failure (2026-04-27 ~11:30 UTC) — Tobias reported wake didn't fire during the live screenshare. Without persisted stdout, root cause was undiagnosable in real-time; hypothesis space included osascript silent error / focus race / macOS Accessibility permission / coalesce-window race — none confirmable.
  2. Bug 2 of #10410 (duplicate-wake delivery) — empirically reproduced multiple times this session (WAKE_SUB:ca08d381 echoed twice for one inbound message; later WAKE_SUB:e5f96999 exhibited same pattern). Root cause undiagnosable from SQLite/process-list alone; the diagnostic split (daemon-side queue/coalesce vs osascript-paste layer) requires the daemon's actual console.log lines ([Bridge Daemon] Delivered ${subscription.id} via osascript to ${appName}) to discriminate.
  3. 3 missed-wake messages in 10:22-10:32 UTC window — daemon healthy (PID stable, lastSyncId advancing, GraphLog had SENT_TO edges) but no wake fired. Hypothesis: silent osascript error caught by deliverDigest's try/catch, logged via console.error then lost when Tobi's terminal scrolled past.

Every wake-failure investigation in this session has been blind beyond what's queryable from SQLite. The daemon's existing console.log/console.error sites are exactly the audit trail we need — but only if they're persisted.

The Problem

ai/scripts/bridge-daemon.mjs writes diagnostic information to process.stdout and process.stderr only. Symptoms:

  • Terminal scrollback is the only audit trail; rotates out as new lines arrive
  • No structured search across delivery history
  • No correlation with lastSyncId advancement state at failure time
  • No cross-session continuity — restart the daemon, lose all prior context
  • Cannot diagnose post-hoc — investigation must happen DURING the failure window or rely on terminal-buffer luck

Existing log sites in the daemon (per current ai/scripts/bridge-daemon.mjs):

console.log(`[Bridge Daemon] Started. Tail-syncing from GraphLog ID: ${lastSyncId}`);
console.log(`[Bridge Daemon] Delivered ${subscription.id} via osascript to ${appName}`);
console.log(`[Bridge Daemon] Delivered ${subscription.id} via tmux to session ${tmuxSession}`);
console.error('[Bridge Daemon] Error in poll loop:', err);
console.error(/* deliverDigest catch path */);

Each line is a high-signal diagnostic anchor. Currently lost the moment the terminal scrolls.

The Architectural Reality

bridge-daemon.mjs is a standalone Node.js script (per AGENTS.md §23 "out-of-process polling script with raw substrate access" convention). It:

  • Imports fs-extra already (used for lastSyncId state file)
  • Has well-defined log directory at .neo-ai-data/wake-daemon/ (the existing lastSyncId location)
  • Runs as a long-lived process; daemon restart resets terminal but NOT the disk-backed state

Substrate options:

  • A. File-based rotating log (e.g., fs.appendFileSync('.neo-ai-data/wake-daemon/bridge.log', line) with daily rotation OR size-based rotation via lightweight library like pino-roll). Sibling-shape to typical Unix daemon logging. Read via view_file or tail -f for live monitoring.
  • B. Route through Neo's logger.mjs — keeps consistency with the Neural Link logger refactor (per #9977). But logger.mjs is for in-process Neo classes; bridge-daemon is a standalone script outside the framework.
  • C. SQLite via shared memory-core graph — sibling to nl_action_log table that RecorderService uses. Possible but heavier (locks, schema migration, query overhead) than the diagnostic use-case warrants.

Recommendation: Option A (file-based with simple rotation). Reasoning:

  • Bridge-daemon is ai/scripts/-convention (raw substrate) per AGENTS.md §23 — file logging fits the directory pattern
  • Diagnostic use-case is read-rare, write-frequent — file is the lighter substrate
  • Live terminal observability preserved (write to BOTH stdout and file)
  • Trivial rotation logic (daily timestamp suffix, or simple max-size + rename) keeps disk usage bounded

The Fix

  1. Add a writeLog(line) helper at the top of bridge-daemon.mjs that wraps both console.log AND fs.appendFileSync('.neo-ai-data/wake-daemon/bridge.log', line + '\n').
  2. Replace existing console.log(...) and console.error(...) sites with writeLog(...) calls.
  3. Add a daily-timestamp rotation: at start of pollLoop, check if today's date differs from the file's last modification day; if so, rename bridge.logbridge.log.YYYY-MM-DD before appending.
  4. Document the log location in the daemon's header comment block.

Optionally:

  • Include lastSyncId value in each delivery log line (already-known context for correlation)
  • Keep the rotation footprint bounded — auto-prune logs older than 30 days

Acceptance Criteria

  • bridge-daemon.mjs writes ALL existing console.log/console.error sites to BOTH stdout (live terminal) AND .neo-ai-data/wake-daemon/bridge.log (persistent audit trail)
  • Daily rotation (or equivalent size-based) keeps individual log files bounded
  • Rotated logs use predictable naming (e.g., bridge.log.2026-04-27) for greppability
  • Log lines include enough structure for post-hoc correlation: timestamp, daemon PID, subscription ID (where applicable), lastSyncId at the time of the line
  • Header comment in bridge-daemon.mjs documents the log path and rotation policy
  • Existing terminal observability (Tobias monitoring stdout in his shell) preserved — no behavior regression
  • Post-merge validation: trigger a wake-failure (e.g., kill osascript mid-delivery, or force a permission denial); confirm the failure mode is captured in the persisted log

Out of Scope

  • SQLite-based logging (Option C) — heavier substrate than diagnostic use-case warrants; defer if/when query-across-deliveries becomes a real workflow need.
  • Centralized agent-wide logger (logger.mjs refactor) — bridge-daemon is a standalone script outside the Neo class system per AGENTS.md §23; pulling the framework logger in cross-cuts the convention.
  • Bug 2 of #10410 root-cause investigation — this ticket creates the substrate; the ACTUAL diagnostic happens after the substrate ships AND a duplicate-wake is captured in the persisted log.
  • Real-time alerting on log patterns — diagnostic substrate is read-rare; alerting is a Phase-2 concern if the failure mode justifies it.

Avoided Traps

  • Reusing logger.mjs — rejected. Bridge-daemon is ai/scripts/-convention (out-of-process), not Neo class system. Pulling the framework logger introduces cross-cutting dependency for a script that should stay light.
  • SQLite for diagnostic logs — rejected. Adds locking + schema migration overhead; query-across-deliveries isn't a current workflow need. File grep / view_file covers the realistic post-hoc audit path.
  • Stdout-only with "just remember to scrollback" — rejected. Multiple empirical instances this session-arc demonstrate the audit trail evaporates the moment investigation is delayed.
  • Per-line buffered async writesfs.appendFileSync is fine; daemon writes are infrequent (coalesce-window-bounded), and synchronous-write preserves audit-trail integrity even if daemon crashes mid-write.

Related

  • Empirical anchor — Bug 2 of #10410 (duplicate-wake delivery) — diagnostic substrate this ticket provides will unblock root-cause investigation
  • Empirical anchor — #10395 Leonard demo wake-failure — without persisted stdout, the failure was undiagnosable in real time
  • Sibling pattern — #9977 routed Neural Link Bridge through logger.mjs to fix stdout corruption (different substrate, related concern)
  • Sibling persistence — RecorderService.nl_action_log in memory-core.sqlite (Neural Link audit trail) — different scope but same architectural intent
  • AGENTS.md §23 sibling-file-lift convention — ai/scripts/ is the "out-of-process polling script with raw substrate access" directory; this ticket's fix stays within that convention

Origin Session ID: 825527f3-06f7-440e-9607-ab37d4a16eb5

Handoff Retrieval Hints

  • query_raw_memories(query='bridge daemon stdout console log persistence wake failure diagnostic audit trail')
  • query_raw_memories(query='Bug 2 duplicate-wake delivery bridge-daemon stdout content')
  • File anchor: ai/scripts/bridge-daemon.mjs (existing console.log/console.error call sites)
  • File anchor: .neo-ai-data/wake-daemon/lastSyncId (existing daemon-state file location; new bridge.log co-locates here)
tobiu referenced in commit 4d6c703 - "feat(bridge-daemon): persist diagnostic logs to disk (#10419) (#10425) on Apr 27, 2026, 4:18 PM
tobiu closed this issue on Apr 27, 2026, 4:18 PM