LearnNewsExamplesServices
Frontmatter
id14396
titleExpose tenant-repo-sync state in deployment diagnostics
stateClosed
labels
enhancementaiarchitecturemodel-experience
assigneesneo-gpt
createdAtJul 1, 2026, 5:04 PM
updatedAtJul 1, 2026, 6:31 PM
githubUrlhttps://github.com/neomjs/neo/issues/14396
authorneo-gpt
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJul 1, 2026, 6:31 PM

Expose tenant-repo-sync state in deployment diagnostics

Closed v13.1.0/archive-v13-1-0-chunk-8 enhancementaiarchitecturemodel-experience
neo-gpt
neo-gpt commented on Jul 1, 2026, 5:04 PM

Context

Cloud deployments can use server-side pull-mode ingestion: the orchestrator owns the tenant-repo-sync task, resolves configured tenantRepos[], mirrors repositories, builds ingestion envelopes, and writes chunks into the Knowledge Base.

A live cloud diagnostics session exposed a blind spot: the public KB/MC diagnostics could prove that the KB collection was healthy but empty, and could show no active/last KB ingestion progress, but could not answer whether the orchestrator had enabled, scheduled, skipped, or failed the tenant-repo-sync task.

Live latest-open sweep: checked the latest 20 open issues at 2026-07-01T15:03:36Z; no equivalent found. Targeted searches for tenant-repo-sync diagnostics, deployment-state bridge tenant repo sync, and compose lookup failures found no equivalent issue. A2A in-flight sweep: checked recent 30 messages; no active overlapping [lane-claim] / [lane-intent] found.

The Problem

When a cloud KB is empty but tenantRepos[] are expected, current diagnostics do not separate these cases:

  • tenant-repo-sync is disabled by deployment gate.
  • Orchestrator is running but the task has never fired.
  • The task fired and skipped because no tenant repos resolved.
  • The task fired and found repos not due because of cadence/jitter/backoff.
  • The task fired and failed during mirror clone, credential resolution, envelope build, embedding, manifest update, or Chroma write.
  • The task succeeded but diagnostics are reading the wrong collection/tenant view.

The existing get_ingestion_progress surface is useful for active/last KB ingestion progress, but it is not enough as the operator-facing answer for orchestrator-owned repo polling. inspect_deployment / get_deployment_state_snapshot expose service state and recovery/heal ledgers, but not the scheduler/task ledger or effective tenant-repo config.

The Architectural Reality

Relevant existing substrate:

  • ai/daemons/orchestrator/scheduling/tenantRepoSync.mjs decides whether the tenant-repo-sync task is due.
  • ai/daemons/orchestrator/scheduling/pipeline.mjs dispatches the service-runner task to TenantRepoSyncService.runTask().
  • ai/daemons/orchestrator/services/TenantRepoSyncService.mjs resolves effective tenantRepos[], applies per-repo due checks, mirrors repos, builds envelopes, ingests chunks, and updates revision state.
  • ai/services/knowledge-base/IngestionService.mjs resolves tenant repo config through graph kb-config:<tenantId> > kb-config.yaml > aiConfig.tenantRepos[].
  • ai/daemons/orchestrator/services/TaskStateService.mjs persists task state, including lastRunAt, lastSuccessAt, lastErrorAt, lastReason, and lastCompletion.
  • ai/daemons/orchestrator/services/DeploymentStateBridgeService.mjs writes the bounded deployment-state snapshot consumed by KB/MC read tools.

The missing surface belongs in the bounded deployment diagnostic layer. It should not require public MCP clients to get shell, Docker, raw logs, credentials, graph write access, or broad file access.

The Fix

Extend the deployment diagnostics with a bounded tenantRepoSync section, either inside inspect_deployment / get_deployment_state_snapshot or as a narrow read-only companion MCP tool.

Minimum useful shape:

  • Orchestrator deployment-mode gate state for tenantRepoSyncEnabled.
  • Scheduler cadence/sweep summary: global cadence, sweep cadence, jitter ratio, and whether the task is currently due.
  • TaskStateService state for tenant-repo-sync: running, lastRunAt, lastSuccessAt, lastErrorAt, lastExitCode, lastReason, lastCompletion / last failure details when bounded.
  • Effective tenant repo config summary: count, redacted tenant ids / repo slugs or hashed identifiers, config tier used (graph, yaml, aiConfig), disabled count, and config-resolution errors.
  • Per-repo bounded state: lastIngestedRev, lastRunAttemptAt, consecutiveFailures, nextDueAt, last outcome code, and last bounded error reason.
  • Clear degraded states when any of the above cannot be read; never silently omit the section.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
inspect_deployment / deployment snapshot DeploymentStateBridgeService.mjs Include a bounded tenantRepoSync diagnostic section Return status: degraded with stable reason codes when task/config state cannot be read learn/agentos/tooling/KnowledgeBaseMcpApi.md, cloud troubleshooting docs Unit coverage for healthy, no-config, not-due, failed, and degraded states
tenant-repo-sync task state TaskStateService.mjs, TenantRepoSyncService.mjs Surface last scheduler/task outcome without raw logs or credentials Preserve current service-only snapshot if task state file is missing, but include the omission as data learn/agentos/cloud-deployment/TenantIngestionModel.md Fixture-backed snapshot tests
Effective tenant repo config summary KnowledgeBaseIngestionService.listConfiguredTenantRepos() Expose counts and redacted identifiers/tier, not secrets If config resolution throws, expose stable error code and message class learn/agentos/cloud-deployment/Configuration.md Tests for graph/yaml/aiConfig tiers and malformed config
Per-repo revision/due state TenantRepoSyncService revision state + isRepoDue() Show enough to distinguish never-run, not-due, backoff, failure, and completed Bound entries and redact repo identity when needed learn/agentos/cloud-deployment/TenantIngestionModel.md Tests for cadence/jitter/backoff projections

Decision Record impact

Aligned with ADR 0014 cloud deployment topology and scheduler task taxonomy. No ADR amendment expected; this is observability for the already-shipped cloud tenant-repo-sync lane.

Acceptance Criteria

  • inspect_deployment or a narrow companion MCP diagnostic exposes tenantRepoSync status.
  • The diagnostic distinguishes at least: disabled, no configured repos, not due, running, completed, failed, and degraded/unreadable.
  • Effective repo config is summarized without secrets and names the winning config tier.
  • Per-repo state exposes last attempt, next due, backoff/failure count, and bounded outcome reason.
  • The surface includes stable reason codes suitable for operator runbooks.
  • Tests cover no-config, configured-not-due, completed, failed clone/credential, failed ingest, and unreadable state file.
  • Docs update the cloud troubleshooting empty-KB section to point at the new diagnostic before recommending manual action.

Out of Scope

  • Exposing raw credentials, clone URLs with secrets, full Docker logs, shell, or lifecycle write authority.
  • Changing the tenant-repo-sync scheduling policy.
  • Replacing push-mode ingestion or bulk import tooling.

Related

  • ai/daemons/orchestrator/services/TenantRepoSyncService.mjs
  • ai/daemons/orchestrator/scheduling/tenantRepoSync.mjs
  • ai/daemons/orchestrator/services/DeploymentStateBridgeService.mjs
  • learn/agentos/cloud-deployment/TenantIngestionModel.md
  • learn/agentos/cloud-deployment/Troubleshooting.md

Origin Session ID: c0dfa949-22de-4daf-bbd2-1e093383fefc

Handoff Retrieval Hints: tenant-repo-sync deployment diagnostics, empty cloud KB tenantRepos not observable, TaskStateService tenant repo sync snapshot

tobiu referenced in commit 1a60bd1 - "feat(ai): expose tenant repo sync diagnostics (#14396) (#14398)" on Jul 1, 2026, 6:31 PM
tobiu closed this issue on Jul 1, 2026, 6:31 PM