who_is_online reports active agents dark during graph-projection lag

Context

who_is_online (shipped via #13527 / #13524, commit 72761b362) computes per-maintainer liveness from add_memory recency — the operator-directed pivot away from the harness beacon so the signal is deployment-agnostic and tenant-safe. Within ~12 min of the post-merge orchestrator restart on 2026-06-19, a V-B-A of the live tool reported every maintainer online: false — including agents that had opened PRs and claimed lanes minutes earlier — and my own last activity as 2026-06-06 (~13 days stale), despite many add_memory saves since.

That is a regression in the feature's core promise (who_is_online liveness must track a working add_memory). This ticket fixes it.

The Problem

add_memory is intentionally decoupled: the WAL append is the never-fail anchor, and the AGENT_MEMORY graph node is projected asynchronously from the WAL (MemoryService._projectMemoryToGraph / _scheduleMemoryGraphProjection). Until projection catches up — and it lags whenever the graph-projection drain is behind or a restart just happened — the just-written turns live only in the WAL, not in the Nodes table.

The canonical recency read (MemoryService.queryRecentTurns) handles this correctly: it queries the graph and overlays _readPendingWalRecencyRows (the not-yet-projected WAL records), so read-after-write stays complete during projection lag.

WakeSubscriptionService._readActivityRecency (the who_is_online recency read) does not overlay the WAL. It reads SELECT MAX(timestamp) FROM Nodes only, so during projection lag it misses every WAL-pending turn and falls back to the agent's last fully-projected node — for me, 2026-06-06. That underreports liveness exactly when add_memory is busiest.

Reproduction (V-B-A, 3 triangulating reads, session abe80be3)

1. get_node('8f134b39…') (my most-recent turn) → null — not in Nodes (WAL-pending).
2. query_recent_turns(@me) → finds it at 2026-06-19T12:41 (via the WAL overlay).
3. who_is_online → reports my last activity as 2026-06-06 (graph-only fallback) and all agents dark.

The Architectural Reality

• ai/services/memory-core/WakeSubscriptionService.mjs:659 _readActivityRecency(owner, nowMs) — graph-only MAX($.properties.timestamp) FROM Nodes with the RLS-OR predicate. No WAL overlay.
• ai/services/memory-core/MemoryService.mjs:1285 queryRecentTurns overlays _readPendingWalRecencyRows({identity, userId, before}) (MemoryService.mjs:826) on top of the graph query — the precedent pattern to mirror.
• ai/services/memory-core/helpers/memoryWalStore.mjs exports readPendingWalRecords({dir, markerType:'graph'}) — the shared primitive both paths can use. WakeSubscriptionService already imports aiConfig (so aiConfig.memoryWal.dir is in reach).
• Graph node shape: {label:'AGENT_MEMORY', properties:{agentIdentity, userId, sessionId, timestamp}} (MemoryService._projectMemoryToGraph:633); WAL records carry the same in record.metadata + graphProjectionVersion:1.

The Fix

In WakeSubscriptionService._readActivityRecency, after the graph MAX query, also read the graph-pending WAL records for owner, apply the same visibility predicate the graph query uses (own-tenant + null-owner + sharedEntity + visibility:team), and return max(graphLatest, walLatest). Mirror the queryRecentTurns precedent (read pending alongside the graph to avoid the projection-completes-mid-read race).

• Touch: ai/services/memory-core/WakeSubscriptionService.mjs (+ import readPendingWalRecords).
• Test: test/playwright/unit/ai/services/memory-core/WakeSubscriptionService.spec.mjs — add a case that seeds a WAL-pending turn (no graph node) and asserts who_is_online reports the agent online. The current seedActivity helper writes graph nodes directly, so the WAL-pending path was never exercised — that is the test gap that let the regression merge.

Contract Ledger

Tool call signature + output shape are unchanged (no wire-format change); this is a behavioral correctness fix to the recency computation.

Decision Record impact

none — aligned-with the WAL-decoupled never-fail add_memory design (this is the read-side completion of that decoupling; the graph projection is derived work that reads must not assume is current).

Acceptance Criteria

• _readActivityRecency overlays graph-pending WAL records (readPendingWalRecords markerType 'graph') and returns the max of graph + WAL recency.
• WAL overlay applies the same RLS visibility predicate as the graph query (no cross-tenant leak in the overlay).
• Unit test: an agent with a WAL-pending-only turn (no projected graph node) reads online: true within the freshness window.
• Unit test: tenant-scoping holds for the WAL overlay (a foreign-tenant WAL-pending turn is NOT visible to a non-owner caller).
• Existing 68 WakeSubscriptionService tests stay green.
• No change to who_is_online's tool signature or output shape.

Out of Scope

• The drain-stall watchdog/alarm (#13551, Vega) — complementary; this fixes the read, that detects the stall.
• Repairing the graph-projection daemon itself — projection lag is inherent to the async-decoupled design; the read must be robust regardless.
• A shared recency-SSOT refactor (extracting one getLatestActivityAt consumed by both queryRecentTurns and who_is_online) — noted as a friction-seed; this fix stays localized to restore correctness first.

Avoided Traps

• Making graph projection synchronous in add_memory — breaks the never-fail decoupling (the whole point of #13527's robustness).
• Depending on the drain being healthy — lag is a normal state (restart, backlog); the recency read must overlay the WAL unconditionally, not assume projection is current.
• Reusing queryRecentTurns wholesale — it fail-closes on no-userId (correct for a cross-session read, wrong for a roster tool that must work in single-tenant stdio); who_is_online keeps its own non-fail-closed RLS-OR visibility policy.

Related

• Origin / regressed-by: #13524, #13527 (commit 72761b362)
• Complementary: #13551 (embed/graph-drain liveness watchdog, Vega)
• Superseded predecessor: #13498 (turn-started beacon — replaced by add_memory-recency)

Release classification: post-release (regression fix in merged coordination tool — non-blocking; boardless per ticket-create §4).

Origin Session ID: abe80be3-6235-4a9e-99bc-b14659ba806a

Handoff Retrieval Hints: query_raw_memories("who_is_online recency WAL pending overlay graph projection lag"); commit anchor 72761b362 (#13527 merge).