LearnNewsExamplesServices
Frontmatter
id14402
titlePreserve GitMirror failure provenance in tenant-repo-sync diagnostics
stateClosed
labels
bugaiarchitecture
assigneesneo-gpt
createdAtJul 1, 2026, 6:58 PM
updatedAtJul 1, 2026, 7:24 PM
githubUrlhttps://github.com/neomjs/neo/issues/14402
authorneo-gpt
commentsCount0
parentIssuenull
subIssues[]
subIssuesCompleted0
subIssuesTotal0
contentTrust
projected
quarantined0
signals[]
blockedBy[]
blocking[]
closedAtJul 1, 2026, 7:24 PM

Preserve GitMirror failure provenance in tenant-repo-sync diagnostics

Closed v13.1.0/archive-v13-1-0-chunk-8 bugaiarchitecture
neo-gpt
neo-gpt commented on Jul 1, 2026, 6:58 PM

Context

During pull-mode cloud-ingestion diagnostics, a concrete failure theory surfaced: configured tenant repositories can be private, and the orchestrator-side repo mirror may receive an access-denied response if the configured credentialRef is missing, stale, or lacks repository read permission.

This is already architecturally plausible and partially implemented:

  • GitMirror.cloneIfMissing() and GitMirror.fetch() inject credentials from credentialRef and throw stable KB_GITMIRROR_* codes on clone/fetch failures (ai/services/knowledge-base/helpers/gitMirror.mjs lines 428-474).
  • TenantRepoSyncService catches per-repo failures, but currently normalizes any non-KB_TENANT_REPO_SYNC_* code to KB_TENANT_REPO_SYNC_SYNC_FAILED before persisting per-repo outcome state (ai/daemons/orchestrator/services/TenantRepoSyncService.mjs lines 484-505).
  • DeploymentStateBridgeService relays lastErrorCode from that already-normalized per-repo outcome (ai/daemons/orchestrator/services/DeploymentStateBridgeService.mjs lines 715-720).

Live latest-open sweep: checked the latest 20 open issues at 2026-07-01T16:57:09Z; no equivalent found. Targeted GitHub searches for tenant-repo-sync GitMirror auth diagnostics credentialRef access denied KB_GITMIRROR returned no open issues. A2A in-flight sweep: checked the latest 30 messages; no competing lane claim for this provenance gap. Semantic KB sweep found the intended GitMirror and deployment-state surfaces, and also exposed the mismatch: KB/docs imply GitMirror failure codes are diagnostic inputs, while live code collapses them before the public snapshot.

The Problem

For a private upstream repository, a deployment can fail before ingestion for several different reasons:

  • credentialRef cannot resolve at runtime.
  • the token resolves but lacks repository read permission;
  • the repository URL is wrong, renamed, deleted, or unreachable;
  • git/network transport fails independently of auth.

Today, operators can see a degraded per-repo state through tenantRepoSync diagnostics, but the structured public surface may only show KB_TENANT_REPO_SYNC_SYNC_FAILED. That is stable, but too broad: it forces the operator back to container logs to distinguish "token/access problem" from "repo missing" or "network blocked". In a cloud deployment, that weakens the diagnostic tools we just added for empty-KB incidents.

The important invariant is not "dump raw git stderr into MCP output". The invariant is "preserve enough non-secret provenance that an operator can branch correctly from the deployment diagnostic snapshot."

The Architectural Reality

The owning surfaces are already well separated:

  • TenantRepoAccessContract keeps cloneUrl credential-free and treats credentialRef as a reference, not secret material.
  • GitMirror owns low-level git clone/fetch/ref failure codes and redacts secret material before exposing errors.
  • TenantRepoSyncService owns per-repo aggregation and outcome state.
  • DeploymentStateBridgeService owns the bounded, public, graph-independent snapshot consumed by KB/MC diagnostics.

The fix belongs in the sync/diagnostic boundary, not in the scheduler. lastErrorCode can remain the stable tenant-sync umbrella code if compatibility needs it, but the lower-level redacted source code needs a bounded field that survives into the bridge snapshot.

The Fix

Preserve redacted underlying error provenance for per-repo tenant-repo-sync failures:

  • In TenantRepoSyncService, keep the existing stable outer lastErrorCode behavior where needed, but also persist a bounded source/subsystem code when the caught error carried one, e.g. lastSourceErrorCode, lastUnderlyingErrorCode, or another clearly named field.
  • Preserve only non-secret categorical fields: source code, source family/subsystem, and optionally a bounded sanitized message class. Do not persist raw stderr, raw clone URLs, tokens, or raw repo identity.
  • In DeploymentStateBridgeService, relay the bounded provenance field through lastOutcome so inspect_deployment / get_deployment_state_snapshot can distinguish GitMirror clone/fetch/credential failures from generic ingest failures.
  • Update docs and tests so the public diagnostic contract matches live behavior.

Contract Ledger Matrix

Target Surface Source of Authority Proposed Behavior Fallback Docs Evidence
TenantRepoSyncService per-repo failure state GitMirror error taxonomy + existing TenantRepoSyncErrors wrapper contract Preserve stable outer lastErrorCode and add a bounded underlying/source error code when available If no source code exists, field is null/omitted and lastErrorCode remains the operator branch point JSDoc in TenantRepoSyncService.mjs Unit test with KB_GITMIRROR_FETCH_FAILED and credential-ref failure
DeploymentStateBridgeService tenantRepoSync.repos[].lastOutcome PR #14398 deployment diagnostics contract Include the bounded source/subsystem error field without exposing raw repo names, clone URLs, tokens, or stderr Snapshot remains backward-compatible with only lastErrorCode Cloud troubleshooting docs / KB MCP docs Unit test on snapshot projection
Cloud troubleshooting docs TenantIngestionModel.md, Troubleshooting.md, Day-0 tutorial Explain how to distinguish missing credential, clone/fetch/auth, and generic ingest failure from structured fields first, logs second Logs remain the fallback only for detailed redacted stderr learn/agentos/cloud-deployment/* Docs lint or focused static check

Decision Record Impact

Aligned with ADR 0014. No ADR amendment expected: this does not change cloud lane classification or scheduler topology; it tightens the diagnostic contract for the existing pull-mode ingestion lane.

Acceptance Criteria

  • GitMirror clone/fetch failures with codes such as KB_GITMIRROR_CLONE_FAILED, KB_GITMIRROR_FETCH_FAILED, and KB_GITMIRROR_CREDENTIAL_REF_INVALID remain redacted and are preserved as bounded source provenance in per-repo tenant-repo-sync outcome state.
  • Existing lastErrorCode compatibility is defined explicitly: either it remains KB_TENANT_REPO_SYNC_SYNC_FAILED with a new source-code field, or a documented migration updates consumers safely.
  • DeploymentStateBridgeService exposes the bounded provenance through tenantRepoSync diagnostics while continuing to hash tenant/repo identity and exclude clone URLs, credential references, tokens, and raw stderr.
  • Tests cover at least one clone/fetch failure, one unresolved credentialRef failure, and the bridge projection of the bounded source code.
  • Docs update the pull-mode troubleshooting path so an operator can distinguish credential/access failures from generic ingest failures without first tailing container logs.

Out of Scope

  • Changing how credentials are stored or injected.
  • Embedding tokens in cloneUrl.
  • Adding live remote credential validation outside the actual git subprocess path.
  • Changing scheduler classification or cadence; that is handled separately.
  • Exposing raw git stderr in public MCP snapshots.

Avoided Traps

  • Do not "fix" this by allowing credential-bearing clone URLs. The clean URL / reference-only credential boundary is correct.
  • Do not add a one-off auth probe that duplicates GitMirror; the mirror worker is already the authority for git transport behavior.
  • Do not make remote diagnostics depend on container log access as the primary path. Logs are fallback detail, not the structured diagnostic contract.

Related

Related: #14396 Related: #14400

Handoff Retrieval Hints

Search terms: tenant-repo-sync GitMirror failure provenance, KB_GITMIRROR_FETCH_FAILED lastErrorCode, DeploymentStateBridgeService tenantRepoSync lastOutcome.

Origin Session ID: c0dfa949-22de-4daf-bbd2-1e093383fefc

tobiu referenced in commit f414dc5 - "fix(ai): preserve tenant repo source error code (#14402) (#14403)" on Jul 1, 2026, 7:24 PM
tobiu closed this issue on Jul 1, 2026, 7:24 PM