LearnNewsExamplesServices
Frontmatter
id14147
titleOrchestrator log hygiene: edge-trigger the deployment-state-bridge success log (it drowns real signal)
stateClosed
labels
bugaimodel-experience
assigneesneo-opus-ada
createdAtJun 26, 2026, 11:35 PM
updatedAtJun 27, 2026, 12:15 AM
githubUrlhttps://github.com/neomjs/neo/issues/14147
authorneo-opus-ada
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJun 27, 2026, 12:15 AM

Orchestrator log hygiene: edge-trigger the deployment-state-bridge success log (it drowns real signal)

Closed v13.1.0/archive-v13-1-0-chunk-7 bugaimodel-experience
neo-opus-ada
neo-opus-ada commented on Jun 26, 2026, 11:35 PM

Context (V-B-A'd against the live orchestrator.log, 2026-06-26)

The operator flagged the orchestrator log being flooded with uber-frequent, no-value lines. Verified against .neo-ai-data/orchestrator-daemon/orchestrator.log (33MB):

  • DeploymentStateBridgeService.mjs:103 logs an INFO line on every successful snapshot write: [DeploymentStateBridge] wrote N service snapshots to ....
  • 2879 such lines since the 12:42 orchestrator start ≈ one every ~11s on average (recent minutes are ~30s, so the writeIntervalMs:30000 gate isn't holding consistently across the run). It is ~half of all recent log lines.
  • The line has no operational value: the snapshot file's own generatedAt + the staleAfterMs watchdog already prove liveness.
  • Demote-to-DEBUG does NOT work: Orchestrator.writeLog (Orchestrator.mjs:563) has no level filter — every line, any level, is appended to disk. The fix must not emit the line on an unchanged snapshot.

Why it matters beyond noise: the heartbeat spam camouflaged a real failure — a KB-sync abort at batch 214/328 (embedder resource could not be found, 20:50) was buried in the same window. Log hygiene here is signal-recovery, not cosmetics.

The fix (scope: DeploymentStateBridgeService.mjs only)

Edge-trigger the success log: compute a signature from the snapshot's services (serviceKey:status set) and emit the INFO line only on first write or when the signature changes (a service appears/disappears or transitions available↔degraded). Steady-state = silent. WARN/ERROR paths unchanged.

Acceptance criteria

  • In steady-state (no service-set / status change), no per-write INFO line is emitted.
  • First write + any service-set or status transition still logs one INFO line.
  • The ERROR path (write failure) is unchanged.
  • Unit coverage: repeated unchanged snapshots → 1 log; a status transition → a second log.

Named follow-ups (separate tickets, not this scope)

  1. Child-stderr default-ERROR misclassificationProcessSupervisor.getChildLogLevel (line 332) defaults unprefixed child lines to ERROR; kbSync progress (✅ Services Ready, Waiting for...) is mis-stamped ERROR. Delicate (it currently surfaces the real batch-214 failure) → careful prefix-contract fix in the child scripts.
  2. Immune-system local/cloud scoping — the 4 immune services (DeploymentStateBridge, ContainerHealthDiagnosis, DataIntegrityDiagnosis, RecoveryActuator) are "both by omission" (in neither localOnly nor cloudOnly). Operator-confirmed direction: keep bridge + data-integrity + B0 in-process recycle local (they provide local value); gate only the B1 docker-socket sibling-container restart to cloud.

Refs #14039 (Agent OS Stability & Self-Healing). Self-assign @neo-opus-ada. Authored by Ada (Claude Opus 4.8, Claude Code).

tobiu referenced in commit 419e094 - "fix(ai): edge-trigger deployment-state-bridge success log (#14147) (#14148) on Jun 27, 2026, 12:15 AM
tobiu closed this issue on Jun 27, 2026, 12:15 AM