LearnNewsExamplesServices
Frontmatter
id14163
titleAutonomous self-heal observability: durable heal-event ledger + queryable immune-system status surface (observe-and-adapt)
stateClosed
labels
enhancementaiarchitecture
assigneesneo-opus-grace, neo-opus-vega
createdAtJun 27, 2026, 1:20 AM
updatedAtJun 28, 2026, 4:46 PM
githubUrlhttps://github.com/neomjs/neo/issues/14163
authorneo-opus-vega
commentsCount1
parentIssue14039
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJun 28, 2026, 4:46 PM

Autonomous self-heal observability: durable heal-event ledger + queryable immune-system status surface (observe-and-adapt)

Closed v13.1.0/archive-v13-1-0-chunk-7 enhancementaiarchitecture
neo-opus-vega
neo-opus-vega commented on Jun 27, 2026, 1:20 AM

Problem

The autonomous immune system (#14132 / #14134 / #14142) records heal runs for rate-limiting (recordRun -> collection-keyed recentRuns) and audits accepted-loss (acceptedLossAuditStore.mjs), but there is no unified, durable, queryable ledger of every heal event (action · collection · trigger-evidence · outcome · timestamp), and no status surface to observe immune-system health.

In cloud (no operator watching), this is the precondition for the operator's "observe the live deployment and adapt" loop -- without it, autonomous heals are invisible after the fact.

Distinct from #14158 (alert on sustained unsafe-input dispatch outcomes -- the alert side). This is the forensic ledger + query side.

Fix shape

  • A durable heal-event ledger: every dispatchHeal outcome (healed / held / deferred / failed) appends {action, collection, evidence-summary, outcome, detail, at} -- compose the existing recordRun / acceptedLossAuditStore rather than a new store where possible.
  • A queryable status surface (NL/MCP tool + optional portal view): recent heal events, per-collection state (healthy / frozen / quarantined), heal-rate trends.
  • Bounded retention (mirror #14128's accepted-loss pruning policy).

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
deployment-state-snapshot.selfHeal createDeploymentStateSnapshot() Additive snapshot field; omitted callers receive selfHeal: null, wired callers pass the bounded immune-status envelope verbatim. null when omitted, preserving existing snapshot consumers. deploymentStateBridgeStore.mjs JSDoc. deploymentStateBridgeStore.spec.mjs: carries the self-heal immune-system status when provided, null by default.
selfHeal status envelope DeploymentStateBridgeService.collectSelfHealSnapshot() Returns {status, source, limit, summary, recentEvents, errors} with source orchestrator-heal-event-ledger; status is available, disabled, or degraded. disabled when no heal-ledger dir is wired; degraded with errors[{reason:'heal-ledger-read-failed', code}] when the ledger read fails. DeploymentStateBridgeService.mjs JSDoc. DeploymentStateBridgeService.spec.mjs: available, disabled, and degraded self-heal snapshot tests.
selfHeal.summary summarizeHealLedger() Folded ledger status {total, byStatus, byType, currentlyFrozen, lastEventAt}; freeze adds a collection to currentlyFrozen, unfreeze removes it, and the set is sorted. null when disabled/degraded; empty ledgers summarize to zero counts, empty currentlyFrozen, and lastEventAt: null. healEventLedgerStore.mjs JSDoc. healEventLedgerStore.spec.mjs summary tests plus bridge snapshot self-heal test.
selfHeal.recentEvents queryHealLedger() via collectSelfHealSnapshot() Newest-first heal-event view, capped by deploymentStateBridge.recoveryRunLimit when finite. [] when disabled/degraded. healEventLedgerStore.mjs and DeploymentStateBridgeService.mjs JSDoc. DeploymentStateBridgeService.spec.mjs: newest-first recent events bounded by recoveryRunLimit.
Read-only/no-heal-trigger guarantee Deployment-state bridge read path The query/status surface reads the heal ledger only; it never appends, dispatches, or triggers a heal. Read failures degrade the envelope and do not perturb the running system. Snapshot still writes with selfHeal.status: degraded or disabled. DeploymentStateBridgeService.mjs read-only comment/JSDoc. DeploymentStateBridgeService.spec.mjs: ledger read failure degrades and does not throw.
Heal-ledger bounded retention appendHealEvent() / pruneHealLedger() Append-only JSONL ledger self-bounds by O(1) stat-gated pruning to newest MAX_HEAL_LEDGER_EVENTS rows once HEAL_LEDGER_PRUNE_TRIGGER_BYTES is crossed. Prune/stat failures are swallowed; next append retries, and heal execution is never gated by observability failure. healEventLedgerStore.mjs JSDoc. healEventLedgerStore.spec.mjs: append-time prune and keep-most-recent retention tests.

Acceptance Criteria

  • Every heal dispatch outcome is recorded durably with {action, collection, evidence, outcome, at}.
  • A read-only query surface returns recent heal events + per-collection immune state.
  • Retention is bounded (pruning policy, sibling of #14128).
  • Read-only -- the ledger/query path never triggers a heal (no embed-canary gating; sibling of #14124).

Scope

v13.1 -- the observe-and-adapt surface for the autonomous immune system. Parent #14039.

Live latest-open sweep: scanned the 25 latest open + the data-integrity cluster (#14132 / #14134 / #14142 / #14158 / #14128); the alert side is #14158, the ledger+query side is unfiled. No duplicate.

Authored by Vega (Claude Opus 4.8, Claude Code). Origin Session ID: 1bb8a27b-ae0d-4668-a9a2-acbbe2387512

tobiu referenced in commit 644643e - "feat(ai): autonomous freeze re-probe substrate — decider + store + cycle orchestrator (#14171) (#14172) on Jun 27, 2026, 2:25 AM
tobiu referenced in commit f0b9854 - "feat(ai): heal-event ledger substrate — durable store + status-summary + query (#14177) (#14178) on Jun 27, 2026, 3:13 AM
tobiu referenced in commit 42afe40 - "docs(adr): amend ADR-0025/0026 escalate terminal → durable async-record (full-self-heal) (#14191) (#14194) on Jun 27, 2026, 9:01 AM
tobiu referenced in commit 2a25c5a - "feat(ai): self-heal observability surface + bounded heal-ledger retention (#14163) (#14272) on Jun 28, 2026, 4:46 PM
tobiu closed this issue on Jun 28, 2026, 4:46 PM