LearnNewsExamplesServices
Frontmatter
id14408
titleinspect_deployment: flag missing snapshot sections
stateClosed
labels
bugaiarchitecture
assigneesneo-gpt
createdAtJul 1, 2026, 8:02 PM
updatedAtJul 2, 2026, 8:31 AM
githubUrlhttps://github.com/neomjs/neo/issues/14408
authorneo-gpt
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJul 2, 2026, 8:31 AM

inspect_deployment: flag missing snapshot sections

Closed v13.1.0/archive-v13-1-0-chunk-8 bugaiarchitecture
neo-gpt
neo-gpt commented on Jul 1, 2026, 8:02 PM

Context

A live cloud empty-KB diagnostic run on 2026-07-01 exposed a narrower follow-up after #14396 / PR #14398. The public KB MCP tools could prove the Knowledge Base collection exists but has count 0, get_ingestion_progress() reported no active or completed source run, and inspect_deployment(staleAfterMs: 600000) returned a fresh deployment-state-snapshot.

The snapshot was fresh but did not include the expected tenantRepoSync top-level section. Local current code and docs now expect that section:

  • ai/daemons/orchestrator/services/DeploymentStateBridgeService.mjs collects tenantRepoSync inside collectSnapshot() and passes it into createDeploymentStateSnapshot().
  • ai/services/memory-core/helpers/deploymentStateBridgeStore.mjs defines tenantRepoSync as a top-level snapshot field.
  • learn/agentos/cloud-deployment/Troubleshooting.md tells operators to inspect snapshot.tenantRepoSync before manually debugging pull-mode empty KBs.

Live latest-open sweep: checked the latest 20 open GitHub issues at 2026-07-01T18:01Z; no equivalent found. Closest related ticket is closed #14396, which added the tenantRepoSync diagnostic section itself. This ticket is deliberately narrower: fail loud when an expected section is absent from a fresh snapshot. A2A in-flight sweep checked the latest 30 messages; no competing [lane-claim] / [lane-intent] for snapshot section completeness was present. Semantic KB ticket sweep for deployment snapshot producer freshness tenantRepoSync schema inspect_deployment orchestrator bridge returned only older deployment/ingestion tickets, not this exact missing-section regression. Exact local content sweep found #14396/#14397/#13926 as related context, not duplicates.

The Problem

A public deployment snapshot can be status: available and fresh while still missing a section that current troubleshooting docs and code treat as required for an empty-KB diagnosis.

That creates a bad operator path:

  1. KB health says Chroma is reachable and the collection count is zero.
  2. KB ingestion progress says no source run is active or recorded.
  3. inspect_deployment returns a fresh snapshot, so the bridge looks current.
  4. The expected tenantRepoSync section is absent, but the reader does not classify that as degraded, stale producer, old schema, or missing required section.
  5. Operators cannot tell whether the orchestrator bridge is stale, an older image is still writing the snapshot, the writer failed before adding the section, or the reader/tool contract is underspecified.

KB-server runtimeFreshness is not enough here. It proves the request-facing KB server matches its checkout/config; it does not prove that the orchestrator bridge producer that writes the snapshot has the same source/config identity or emits the expected schema sections.

The Architectural Reality

  • DeploymentStateBridgeService.collectSnapshot() owns producer-side composition of the bounded snapshot.
  • createDeploymentStateSnapshot() owns the top-level snapshot envelope and currently accepts tenantRepoSync = null.
  • readDeploymentStateSnapshot() only checks file presence, size, JSON parse, and age. It returns status: available for any fresh JSON with a finite generatedAt, even when expected sections are absent.
  • inspect_deployment / get_deployment_state_snapshot are the public read-only proof surfaces. They should not gain shell, Docker, or lifecycle authority; the fix belongs in bounded schema/provenance validation and producer metadata.
  • #14396 asked for tenantRepoSync to exist and never be silently omitted. The live failure mode shows the read path still needs a generic completeness/provenance guard so missing sections become data, not absence.

The Fix

Add a schema-completeness/provenance layer for deployment snapshots:

  • Define the expected top-level sections for the current snapshot schema, at least services, bridgeDiagnostics, recoveryRuns, selfHeal, and tenantRepoSync.
  • Have the read path classify a fresh snapshot with missing required/current sections as degraded or available-with-warnings using stable reason codes such as snapshot-section-missing or snapshot-producer-schema-stale.
  • Include bounded producer metadata in snapshots so readers can distinguish KB-server freshness from orchestrator bridge producer freshness. Useful fields may include bridge producer name, bridge schema/features, code/config digest if already available in the orchestrator, and feature flags for emitted optional sections.
  • Keep backward compatibility explicit: older snapshots may still be readable, but the tool must surface that they do not satisfy the current diagnostic contract.
  • Update cloud troubleshooting docs so absence of tenantRepoSync has a first-class branch: verify orchestrator image/restart, bridge writer source/schema, and snapshot producer freshness before chasing Git credentials or embeddings.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
readDeploymentStateSnapshot() result ai/services/memory-core/helpers/deploymentStateBridgeStore.mjs Fresh but incomplete snapshots carry stable degraded/warning status and missing-section details Older snapshots remain readable but not silently treated as complete Cloud troubleshooting docs Unit tests for missing tenantRepoSync, missing bridgeDiagnostics, and valid current snapshot
Deployment snapshot producer metadata DeploymentStateBridgeService.mjs + snapshot helper Snapshot includes bounded producer/schema/features metadata proving which bridge writer emitted it If producer metadata is absent, reader reports legacy/unknown producer Cloud deployment diagnostics docs Unit test fixture for legacy snapshot vs current snapshot
inspect_deployment / get_deployment_state_snapshot public tool contract KB/MC MCP read tools over the bounded bridge file Tool output points operators at producer/schema mismatch when expected sections are absent No Docker/socket/shell authority is exposed learn/agentos/cloud-deployment/Troubleshooting.md MCP/tool-level unit or fixture test asserting surfaced reason codes

Decision Record impact

Aligned with ADR 0014 cloud deployment topology and the existing graph-independent deployment-state bridge. No ADR amendment expected. This tightens the diagnostic contract for the public read path; it does not change orchestration authority or add new runtime privileges.

Acceptance Criteria

  • A fresh snapshot missing tenantRepoSync no longer returns as an unqualified available diagnostic result.
  • Missing expected top-level sections are reported with stable reason codes and section names.
  • Snapshot output includes bounded producer/schema/features metadata sufficient to distinguish KB-server freshness from orchestrator bridge writer freshness.
  • Legacy snapshots remain readable but are classified as legacy/incomplete instead of silently satisfying the current contract.
  • Tests cover a complete current snapshot, a legacy snapshot without producer metadata, and a fresh incomplete snapshot missing tenantRepoSync.
  • Docs update the empty-KB troubleshooting branch for the case where snapshot.tenantRepoSync is absent.

Out of Scope

  • Re-adding the tenantRepoSync section itself; that was #14396 / PR #14398.
  • Exposing Docker socket, shell, restart, or arbitrary runtime state through public MCP tools.
  • Changing tenant repo credential semantics.
  • Changing scheduler cadence or heavy-maintenance classification.
  • Naming any private tenant or deployment.

Related

  • #14396 — exposed tenant-repo-sync state in deployment diagnostics.
  • #14397 — explained deployment compose-service lookup failures.
  • #14400 — resolved tenant-repo-sync scheduler classification drift.
  • #14402 — preserved GitMirror failure provenance in tenant-repo-sync diagnostics.
  • #14404 — graph-only tenant repo config discovery.
  • ai/daemons/orchestrator/services/DeploymentStateBridgeService.mjs
  • ai/services/memory-core/helpers/deploymentStateBridgeStore.mjs
  • learn/agentos/cloud-deployment/Troubleshooting.md

Handoff Retrieval Hints: deployment snapshot missing tenantRepoSync section, inspect_deployment producer freshness schema completeness, deploymentStateBridgeStore missing section degraded result.

tobiu closed this issue on Jul 2, 2026, 8:31 AM
tobiu referenced in commit b667067 - "fix(ai): flag incomplete deployment snapshots (#14408) (#14409)" on Jul 2, 2026, 8:31 AM