LearnNewsExamplesServices
Frontmatter
id15640
titleai:restore: preflight the embedding provider before re-ingest
stateClosed
labels
enhancementai
assigneesneo-kimi-phoebe
createdAtJul 21, 2026, 6:44 PM
updatedAtJul 22, 2026, 11:46 AM
githubUrlhttps://github.com/neomjs/neo/issues/15640
authorneo-kimi-phoebe
commentsCount1
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJul 21, 2026, 8:07 PM

ai:restore: preflight the embedding provider before re-ingest

Closed Backlog/active-chunk-8 enhancementai
neo-kimi-phoebe
neo-kimi-phoebe commented on Jul 21, 2026, 6:44 PM

Context

Backup bundles capture logical state (JSONL exports of the Chroma collections), not the physical HNSW indexes — restore re-ingests and re-embeds through the configured embedding provider (ai/scripts/maintenance/restore.mjsmanageDatabaseImport SDK path).

On a fresh host, the provider may not be ready: the embedding model has never been pulled (e.g., Ollama starts empty and needs ollama pull <embedding-model>), or the provider daemon is down. Today's failure shape is a stall deep inside the import — the worst kind: the restore appears to run, no actionable error, and the operator learns nothing about the actual prerequisite.

Live latest-open sweep: checked latest 20 open issues at 2026-07-21 ~16:45Z; no equivalent found. A2A in-flight claim sweep (last 60 min): no competing claim.

The Problem

restore.mjs validates the bundle (torn/partial fails fast — good) but never validates the environment it must re-embed through. The embedding-provider reachability/model-presence check only surfaces as a timeout or a silent hang during the first embed batch.

The Architectural Reality

  • The embed-write canary exists already: the Memory Core healthcheck probes the active embedding provider with a write canary (embeddingWriteCanaryTimeoutMs, raised 5s→30s for cold embedders in #14182). The probe pattern is proven substrate.
  • Provider config: NEO_EMBEDDING_PROVIDER / NEO_OLLAMA_HOST / NEO_OLLAMA_EMBEDDING_MODEL env-bound leaves (ADR-0019).
  • restore.mjs pre-flight section already does bundle integrity + topology checks BEFORE any write — the natural home for one more pre-flight.

The Fix

Add an embedding-provider pre-flight to restore.mjs, before any import write:

  1. Resolve the active embedding provider from config.
  2. Probe: minimal embed call against the provider (reuse the canary's probe shape; short timeout, e.g. 10s).
  3. On failure, refuse with an actionable error naming the provider, host, model, and the exact remediation (e.g., ollama pull <model>), and exit non-zero before touching any store.
  4. --skip-embed-preflight escape hatch for exotic providers the probe can't cover (documented, logged).

Sibling precedent: mcpHealthcheck.mjs and the embed canary in the healthcheck path — same probe-and-fail-fast discipline.

Acceptance Criteria

  • Restore against an unreachable/absent embedding provider refuses before any write, with provider + model + remediation in the error
  • Restore with a healthy provider proceeds unchanged (no behavior delta)
  • The escape hatch is honored and logged
  • Unit specs cover refuse-before-write and pass-through paths with a stubbed provider

Out of Scope

  • First-boot restore automation (separate ticket)
  • Provider-side model auto-pulling (provider-specific orchestration, not a restore concern)

Related

  • #10871 (backup/restore parity), #14182 (canary timeout for cold embedders), #14162 (read-only diagnostics exempt from the embed-canary gate)

Origin Session ID: d8a51237-4fcc-4171-8071-a391da0be361

Retrieval Hint: "restore embedding provider preflight re-ingest stall cold ollama"

tobiu referenced in commit ac6e271 - "fix(ai): ai:restore preflights the embedding provider before re-ingest (#15640) (#15643) on Jul 21, 2026, 8:07 PM
tobiu closed this issue on Jul 21, 2026, 8:07 PM