What is the Neural Link?

The Neural Link is a bi-directional bridge that connects AI agents directly to the Neo.mjs runtime. It lets agents inspect the Scene Graph, component state, event listeners, computed styles, and DOM rectangles, and mutate the running application in real time.

Why is Neo.mjs called an Application Engine instead of a framework?

Neo.mjs maintains persistent application objects in a worker-backed Scene Graph instead of compiling application state away into ephemeral DOM nodes. That architecture enables multi-window orchestration, runtime permutation, and deep AI introspection.

What is Context Engineering?

Context Engineering shapes the information and tool environment around AI agents. Neo.mjs implements it through Knowledge Base, Memory Core, GitHub Workflow, and Neural Link MCP servers for frontier harnesses, plus a File System MCP server for internal Neo.ai.Agent local loops.

What is the Neo.mjs Agent OS?

The Neo.mjs Agent OS is the repository Brain: source code and services for Memory Core, Knowledge Base, Active Hybrid GraphRAG, DreamService, Golden Path synthesis, A2A coordination, and Neural Link tooling.

Frontmatter

id	16348
title	A failed backup must not forfeit a full day of cadence
state	Closed
labels	bugai
assignees	neo-opus-grace
createdAt	Aug 2, 2026, 2:32 PM
updatedAt	Aug 3, 2026, 12:30 PM
githubUrl	https://github.com/neomjs/neo/issues/16348
author	neo-opus-grace
commentsCount	12
parentIssue	null
subIssues	[]
subIssuesCompleted	0
subIssuesTotal	0
contentTrust
projected
quarantined	0
signals	[]
blockedBy	[]
blocking	[]
closedAt	Aug 3, 2026, 12:30 PM

A failed backup must not forfeit a full day of cadence

Closed Backlog/active-chunk-12 bugai

neo-opus-grace commented on Aug 2, 2026, 2:32 PM

Context

Operator direction 2026-08-02, after a live incident. Two bundles produced today prove the same defect from opposite ends, and the second one blocked the deploy guard until I removed it by hand.

Live latest-open sweep 2026-08-02T12:31:54Z. Neighbours checked: #15641 (CLOSED — built the off-host sync hook), #16055 (CLOSED — the survivability incident), #16208 (Chroma persist path). None owns the success/failure verdict of a capture.

The Problem

The backup lane can write a bundle that captured nothing and record it as a success.

Specimen 1 — backup-2026-07-31T04-57-18.233Z, 64K, still on disk:

"mc": { "message": "Export complete. Exported 0 memories, 0 summaries, 0 temporal summaries", "count": 0,
        "memories": { "backupFile": null, "expected": 0, "exported": 0 } }
"kb": { "message": "Export complete. Exported 0 knowledge base chunks.", "count": 0 }

It ran mid-cutover while the Chroma re-embed was still in flight, found an empty plane, and reported "Export complete." The subsystem contract has no way to say "the source was unavailable" — expected: 0, exported: 0 is byte-identical to a genuinely empty corpus, so a dependency outage and a legitimately empty plane produce the same receipt.

Specimen 2 — backup-2026-08-02T05-12-55.917Z, deleted 12:30Z under operator authorization:

7 empty directories + kb/knowledge-base-backup-…jsonl  (0 bytes)   NO bundle-meta.json
directory mtime 05-12-55.917Z   file mtime 05-12-55.930Z    (13 ms apart)

The orchestrator was crash-looping (325 restarts, deadlocked on its own authority lease — see #16230 / #16242). The lane opened the KB output stream and died before the first byte. It left a bundle-shaped directory behind.

Both then compound with newest-only selection. verifyLatestBackupRestorable (ai/scripts/maintenance/restore.mjs:750-760) sorts bundle names reverse-lexically and inspects only bundleNames[0]. Specimen 2 was the newest, so the guard reported no verified bundle while a complete 5.0 GB bundle sat directly beside it:

before  →  (guard blocked by the 0-byte artifact)
after   →  restorable: true   code: RESTORABLE   rowTotal: 94325
           bundleRoot: backup-2026-08-01T12-13-23.398Z

One aborted run made 94,325 recoverable rows invisible, and the repair was rm -rf on a directory.

The Architectural Reality

ai/scripts/maintenance/backup.mjs — writes per-subsystem receipts into bundle-meta.json. The receipt records what was exported; it does not record what should have been exportable, so shortfall is unrepresentable.
ai/scripts/maintenance/restore.mjs — verifyLatestBackupRestorable already rejects a zero-row bundle with BUNDLE_EMPTY, and its comment names this exact incident class. That guard is correct and is not the gap. It runs at restore/preflight time; nothing applies the same standard at capture time, and nothing looks past the newest bundle.
backup is a priority-zero orchestrator lane (ai/daemons/orchestrator/scheduling/pipeline.mjs:13), cadence backupMs = DAY_MS. A failed run therefore costs a full day by default.

The Fix

(Prescription — the operator named several viable options; this ticket should choose one deliberately, not fold them together.)

A capture that cannot reach its sources is a failure, not an empty success. Options, with trade-offs:

Refuse to finalize. If a subsystem's source is unreachable, do not write a bundle directory at all. Cleanest invariant — every directory under the backup root is then a real attempt — but it discards partial captures that might still have value.
Wait for the dependency, then proceed. Chroma down ⇒ block and retry within the run. Keeps the daily slot, but an unbounded wait inside a priority-zero lane is its own hazard and needs a cap.
Raise scheduler priority and re-try. Record the failure, promote the lane so the next attempt is soon rather than in DAY_MS. Best fit for the crash-loop specimen, where the dependency was fine and the process died.
Mark the bundle failed and keep it. Write bundle-meta.json with an explicit failure verdict so the artifact is self-describing rather than merely thin.

Option 3 plus the failure receipt from 4 is my read — a lane that dies should not surrender its daily slot, and a bundle that failed should say so rather than be inferred from its size. But 1 has the cleanest invariant and the choice belongs in this ticket.

Independent of which is chosen: selection must fall back past an unusable newest bundle to the newest valid one, or a single bad run keeps hiding good history.

Acceptance Criteria

A capture whose source is unreachable does not produce an artifact that reads as a successful backup.
expected vs exported can express shortfall — an unreachable source is distinguishable from a legitimately empty corpus in the receipt.
A run that dies mid-write leaves either nothing or a self-describing failed artifact; never a bundle-shaped directory with no bundle-meta.json.
Restorability selection falls back to the newest valid bundle rather than stopping at the newest. A spec covers: valid bundle + newer invalid bundle ⇒ RESTORABLE against the valid one.
A failed run does not silently forfeit its cadence slot — retry policy is explicit and bounded.
Spec coverage for both live specimens: source-unreachable (zero rows exported) and abort-mid-write (no meta).
Post-merge: with a deliberately unreachable Chroma, the lane reports failure and the previous good bundle remains the verified one.

Contract Ledger

Backfilled per PR #16421 review RA-4. Every row below is read from origin/dev at the implementing head, not restated from the prescription.

Target Surface	Source of Authority	Proposed Behavior	Fallback	Docs	Evidence
`orchestrator.intervals.backupRetryDelayMs` (`NEO_ORCHESTRATOR_BACKUP_RETRY_DELAY_MS`, `number`, default `900000`)	`ai/configBase.mjs` leaf; ADR-0019 read-at-use-site	Minimum spacing between retries of a failed backup run	`0` disables the retry path entirely; the periodic sweep behaves exactly as before the feature	Leaf JSDoc in `ai/configBase.mjs`	`either retry value at 0 restores the pre-retry behaviour` (with a positive control)
`orchestrator.intervals.backupRetryWindowMs` (`NEO_ORCHESTRATOR_BACKUP_RETRY_WINDOW_MS`, `number`, default `3600000`)	same	How long the retry window stays open, measured from `failureStreakStartedAt`	`0` disables the retry path entirely	same	same spec, second branch
Effective attempt budget	derived, not declared	`floor(backupRetryWindowMs / backupRetryDelayMs)` = `4` at defaults	n/a	Leaf JSDoc	asserted by running the clock over an always-failing lane, not by restating the formula
`TaskStateService` task state `failureStreakStartedAt` (ISO string \| `null`)	`ai/daemons/orchestrator/services/TaskStateService.mjs`	Opens at the FIRST failure after a success (`??=`), preserved across every later failure, cleared by `markCompleted()`. Opened by all three terminal-failure writers through one shared `openFailureStreak()` transition — `markFailed()` (exited non-zero), `markSpawnFailed()` (spawn threw), and `readState()`'s interrupted-run normalization	absent/unparseable ⇒ treated as no open streak ⇒ no retry	Method JSDoc + `openFailureStreak()` JSDoc	`failureStreakStartedAt opens at the first failure and never slides` (mutation to `=` fails it); `markSpawnFailed opens the streak on a known-good lane`; `a synchronous spawn failure leaves the lane retry-due, not healthy` — driving the real `ProcessSupervisorService.runTask()` with a throwing `spawnFn`
`TaskStateService` task state `interruptedAt` (ISO string \| `null`)	same	Stamped by `readState()` when a persisted `running: true` is normalized, and committed to disk by `configure()` before any consumer can read the lane; cleared by `markCompleted()`. Without the commit the crashed bytes survive and each restart re-derives a fresh anchor, sliding the bound once per outage	absent ⇒ no interruption recorded	Method JSDoc	`readState normalizes an interrupted run fail-closed`, with a cleanly-stopped positive control; `the interrupted-run anchor is persisted, so a second restart cannot slide it` — two consecutive `configure()` boots over one persisted crash record, asserting persisted bytes and an unchanged `windowEndsAtMs`
Deployment-state snapshot `maintenance.retry`	`ai/daemons/orchestrator/services/DeploymentStateBridgeService.mjs`	`{phase, retriesRemaining, windowEndsAtMs, streakStartedAtMs, interruptedAt}`, additive	Omitted entirely when no backup task state is supplied, so a detached projection keeps its prior shape	`describeBackupRetryState` JSDoc	orchestrator + off-host-sync suites green
`maintenance.retry.phase` vocabulary	`BACKUP_RETRY_PHASE` (frozen)	`healthy` (no open streak, has succeeded) · `retrying` (streak open, window open) · `exhausted` (streak open, window closed) · `unanchored` (never succeeded, no streak)	unparseable timestamps degrade to `unanchored`, never to a stray epoch	JSDoc on the frozen map	one spec per phase, plus the unparseable-timestamp guard
`maintenance.retry.retriesRemaining`	`countRemainingRetries`	The count of retries that will actually fire before the window closes — next firing at `max(lastRunAt + delay, now)`, then one per delay, strictly before the window end	`0` whenever either retry value is disabled or no streak is open	JSDoc	swept at five offsets across the delay against a clock simulation; a wall-clock `floor()` under-reports at retry-due and over-reports at the boundary

Why the anchor is failureStreakStartedAt and not lastSuccessAt: a periodic run fails roughly one backupMs after the last success, so a window anchored on the success is already expired before the first retry can be spaced — the policy silently no-ops at its own shipped defaults. It also must not slide: lastErrorAt advances with every retry, and on a lane that wins its scheduling pick unconditionally an unclosable window is a heavy-lease monopoly. Opening at the failure and never moving is what satisfies both.

Why one shared transition rather than three writers: making the anchor the sole activation fact makes every producer of "failure" load-bearing, and a writer that records lastErrorAt without opening the streak is a silent forfeit of the entire budget rather than a reporting gap. markSpawnFailed() was exactly that writer. clearRecovered() was examined as a fourth candidate and ruled out: it is reachable only after adoptRunning() at boot, which always follows the interrupted-run normalization, and it never clears the anchor.

Out of Scope

Manual npm run ai:backup as a remedy. Operator direction 2026-08-02: manual invocation is an anti-pattern, and whether the lane should be orchestrator-triggered only is an open question this ticket should not settle by implication.
The orchestrator crash-loop itself — #16230 / #16242, the single-owner authority-lease invariant.
The off-host sync posture (#15641, shipped) and its reporting.
redeployPreflight's --initialize guard — #16344.

Avoided Traps

Treating the guard as the defect. verifyLatestBackupRestorable correctly rejects zero-row bundles and its comment already names this incident class. The gap is that capture applies no equivalent standard, and that selection stops at the newest.
Reading "Export complete" as a lie. It is not — it accurately reports that the export finished. The contract simply cannot express "finished having found nothing because the source was gone."
Fixing this by deleting bad bundles. That is what I did by hand today, under operator authorization, to unblock the guard. It restored rowTotal: 94325 in seconds — which is precisely why it must not be the standing remedy.

#16230 / #16242 — the orchestrator authority-lease deadlock that produced specimen 2
#16055 — the survivability incident whose shape this reproduces
#15641 — the off-host sync hook (shipped; different layer)
#16344 — the preflight --initialize guard
D#16304 — delivery mechanism; the guarded deploy path depends on this verdict

Origin Session ID: 713db0da-2239-44ea-ba5b-931be90d34fc

Retrieval Hint: query_raw_memories("backup exported zero memories reported Export complete newest-only bundle selection")