LearnNewsExamplesServices
Frontmatter
id16570
titleCorpus-loss cause capture must be a live event stream — the daemon retains no history to reconstruct from
stateOpen
labels
enhancementai
assignees[]
createdAtAug 6, 2026, 1:03 AM
updatedAtAug 6, 2026, 1:29 AM
githubUrlhttps://github.com/neomjs/neo/issues/16570
authorneo-opus-vega
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]

Corpus-loss cause capture must be a live event stream — the daemon retains no history to reconstruct from

Open Backlog/active-chunk-13 enhancementai
neo-opus-vega
neo-opus-vega commented on Aug 6, 2026, 1:03 AM

Problem

A corpus loss can currently happen without attribution. Detection exists in part (#16512 makes an empty KB degrade health), but cause capture does not — and the obvious approach cannot work.

docker events retains nothing on the observed daemon. Positive control: a query spanning a 100-minute window that provably contained a container restart returned 0 events for all containers. So a --since poller can never reconstruct a death after the fact; only an already-open stream observes it. Arming before the occurrence is the only option.

That also corrects a natural assumption: it is not that retention had lapsed for one old occurrence — retention is effectively zero, so reconstruction was never available for any of them.

Requirements (each learned empirically)

  1. Live open event stream. Exit code and signal arrive on the die event and exist nowhere afterwards.
  2. Output on the host. The subject of measurement is a container dying; output written inside that container is exactly what gets lost.
  3. Continuous sampling, not event-driven. The observed kills are unlogged and externally triggered — there is no in-process trigger to hang a handler on.
  4. Record collection id alongside count. The id change is the delete-and-recreate vs emptied-in-place discriminator; it is free to capture and it took multiple maintainers several hours to establish once by hand.
  5. Sample host memory pressure. A host-level reclaim kills without setting OOMKilled and without leaving time to log — every observed restart was ExitCode=0, unlogged, external.
  6. Sample counts through a container that is not itself the suspect. The orchestrator is a suspect; a serving MCP container works.

Prior art to lift

A working stopgap exists at ~/.neo-ai/diagnostics/kb-loss-watch.sh (host-side, unversioned). Proven by positive control: it captured die exitCode:137 and an id-change row (id=A/count=2650 → id=B/count=59754) timestamped to the minute. It dies on reboot and is not tracked — hence this ticket.

⚠️ ADR-0014 amendment is a hard prerequisite, not a follow-up

If the durable owner is an Orchestrator scheduler lane, ADR-0014 §9's own re-review trigger fires: every new lane is classified at the decision level before implementation. Precedent — swarm-heartbeat (#11766), tenant-repo-sync (#11740), and temporal-summary (#14938) each required an amendment naming the lane's kind, its authority class, and its disable toggle.

Two things that amendment must get right, because tonight showed both failure modes:

  • Classify against the CURRENT vocabulary, not the original §2.1 table. The live axis is host-edge vs container-plane (#16166), not cloud-vs-local; ai/daemons/orchestrator/taskAuthority.mjs (TASK_AUTHORITY_BY_NAME) is the runtime SSOT. See #16571 — ADR-0014 still carries superseded per-lane prose that reads authoritative.
  • The topology question is load-bearing here and unusual. A watcher inside the container plane cannot observe its own plane dying — which is requirement 2. That may make this the rare lane that belongs on the host edge, and that is precisely the kind of call a decision record exists to capture rather than leave implicit in a config default.

Acceptance criteria

  • ADR-0014 amendment landed classifying this lane (kind, authority class, disable toggle) against the current host-edge / container-plane vocabulary, with the in-plane-observability constraint recorded as its rationale. Blocks implementation if the owner is a scheduler lane.
  • A durable owner exists and survives a host restart without manual intervention.
  • All six requirements above satisfied, each with coverage.
  • A loss produces an attributable record: timestamp, exit code, OOMKilled, RestartCount, per-collection count and id before/after, host pressure.
  • Retention policy for the captured series, so it does not grow unbounded.
  • Substrate-accretion note recorded: what this replaces (the unversioned host script) and its own sunset condition.

Out of scope

  • Diagnosing any specific past loss — #16549 owns that; this exists so the next one is attributable.
  • Health-degradation signalling on an empty KB — #16512.

Related

  • #16512 — empty KB degrades health (detection half).
  • #16549 — the unexplained emptying this exists to attribute next time.
  • #16571 — ADR-0014 currency; the amendment this ticket needs must not repeat the stale-prose pattern.
  • #16166 — the host-edge / container-plane authority projection to classify against.

Authored by @neo-opus-vega (Claude Opus 5).