Sub 2 Phase 2 — REM run/stage state model JSONL writer + per-phase wall-clock + cadence-overflow detection

Parent context

Epic #12065 Sub 2 #12068 — Phase 2 follow-up to PR #12081 (Phase 1a) + Phase 1b ticket. Phase 1a shipped the 5 axis-count helpers; Phase 1b exposes them via MCP. Phase 2 ships the per-cycle JSONL state model that captures run/stage outcome + per-phase wall-clock + cadence-overflow detection (operator-framed regression-prevention substrate).

Dependency

BLOCKED-ON:

1. PR #12081 merge (Phase 1a helpers must exist for state model to invoke them)
2. Phase 1b PR merge (MCP exposure scaffold must exist for AC-WallClock-4 trending surface)

Operator framing (anchor)

@tobiu 2026-05-27 ~08:00Z:

"if we trigger the process / task inside the orchestrator every hour => if it e.g. was 10m before and now magically grew to 90m (>the 60m repeat window) => trouble."

Per-phase wall-clock + cadence-overflow detection is the regression-prevention substrate for this exact failure mode.

Contract Ledger

Per ticket-intake-workflow.md §7 — documented upfront per ticket-author discipline.

JSONL state model schema

Storage: append-only JSONL at .neo-ai-data/rem-runs/<runId>.jsonl (durable across restart; queryable post-incident). Per Discussion #12062 §2.11 + scope-extended per #12068 comment 4552584285.

// Cycle-level entry (one per cycle invocation)
{
  runId                 : String,   // crypto.randomUUID()
  reason                : String,   // 'manual' | 'orchestrator-periodic' | 'startup-auto'
  startedAt             : Number,   // epoch ms
  completedAt           : Number,   // epoch ms
  wallClockMs           : Number,   // completedAt - startedAt (derived; convenience field)
  configuredCadenceMs   : Number,   // aiConfig.orchestrator.intervals.dreamMs (resolved at cycle start)
  cycleOverflowSignal   : Boolean,  // wallClockMs > configuredCadenceMs * overflowThreshold (default 0.8)
  cycleOverflowRatio    : Number,   // wallClockMs / configuredCadenceMs — sortable trending signal
  outcome               : String,   // 'completed' | 'skipped' | 'failed'
  reasonCode            : String,   // 'ok' | 'no-undigested-sessions' | 'already-processing' | 'provider-unreachable' | 'backpressure' | 'lease-held' | 'extraction-failed' | 'topology-failed' | 'lazy-edge-drain-failed'
  failurePhase          : String,   // populated when outcome === 'failed'
  failureReason         : String,
  lastSuccessfulPhase   : String,
  cycleScopePhases      : [String], // ordered list of phases attempted
  perPhaseStates        : [
    {
      phase            : String,    // e.g. 'providerReady', 'sessionQuery', 'memorySessionIngest', 'triVector', 'topology', 'gapSession', 'lazyEdges', 'garbageCollection', 'goldenPath'
      startedAt        : Number,
      completedAt      : Number,
      wallClockMs      : Number,
      status           : String,    // 'completed' | 'skipped' | 'failed'
      details          : Object     // phase-specific telemetry
    }
  ],
  perSessionStates      : [
    {
      sessionId        : String,
      payloadSizeTokens: Number,
      memorySessionIngest: {status, errorReasons},
      triVector        : {status, attempts, errorKind?},
      topology         : {status, conflictCount},
      gapSession       : {status},
      graphDigestedFlag: Boolean,
      failureReasons   : [String]
    }
  ]
}

Wall-clock + cadence-overflow contract

MCP exposure delta (extends Phase 1b)

Retention

Acceptance Criteria

• AC1: JSONL state model writer ships — invoked from either processUndigestedSessions (current production path) OR Sub 3 executeRemCycle if landed first
• AC2: Per-phase + cycle-level wallClockMs derived from epoch-ms timestamps; reported regardless of outcome
• AC3: Per-session state model captures payload size + per-stage outcome (memorySessionIngest, triVector, topology, gapSession, graphDigestedFlag)
• AC4: Topology outcome included in state model (closes silent-failure surface of PR #12077 runbook hypothesis #10 — TopologyInferenceEngine void-return ambiguity)
• AC-WallClock-1: Per-phase entries include startedAt + completedAt + derived wallClockMs
• AC-WallClock-2: Cycle entry includes cumulative wallClockMs + configuredCadenceMs + derived cycleOverflowSignal + cycleOverflowRatio
• AC-WallClock-3: WARN log when cycleOverflowSignal: true fires — operator-visible signal before back-to-back overlap actually happens
• AC-WallClock-4: Phase 1b MCP get_rem_pipeline_state tool extended with recentCycles field surfacing wall-clock + overflow trending
• AC-WallClock-5: Unit test seeds fake state model with wallClockMs > configuredCadenceMs * 0.8 + asserts cycleOverflowSignal: true + WARN log emitted (mocked logger)
• AC6: Documentation update at learn/agentos/rem-state-model.md (created in Phase 1b) appended with state-model schema + cadence-overflow semantics + operator-cookbook examples
• AC7: Live re-measurement of 5 axes + wall-clock trending against operator's production substrate post-merge (operator-side empirical validation)

Avoided Traps

• ❌ Don't ship Phase 2 BEFORE Phase 1a (#12081) + Phase 1b merge — schema readers depend on the helpers + the MCP tool
• ❌ Don't enforce JSONL disk cap in Phase 2 — adds operator-policy concerns out of scope; defer to Phase 3 if file count grows
• ❌ Don't bundle Sub 3 executeRemCycle integration into this ticket — Sub 3 #12069 owns that integration; this Phase 2 ships the state-model writer + invocation from current processUndigestedSessions; Sub 3 absorbs/extends later
• ❌ Don't use graph nodes for state model — graph mutations during REM run risk active-control-plane collateral damage per Discussion #12062 §2.10. JSONL files are off-graph + durable.
• ❌ Don't auto-rotate / auto-purge JSONL — same operator-policy boundary as backups; manual tooling existing per ai:backup family

Related

• Epic #12065
• Sub 2 ticket #12068
• Sub 2 Phase 1a PR #12081 (dependency)
• Sub 2 Phase 1b ticket #12087 (dependency)
• PR #12077 Sub 1 forensics runbook (hypothesis #10 — TopologyInferenceEngine void-return ambiguity that AC4 closes)
• Discussion #12062 §2.11 (state model schema source)
• Sub 3 ticket #12069 (eventual consumer of state model)

Operator framing locked

"if we trigger the process / task inside the orchestrator every hour => if it e.g. was 10m before and now magically grew to 90m (>the 60m repeat window) => trouble." — @tobiu 2026-05-27 ~08:00Z

This ticket is the regression-prevention substrate.