Context
During pull-mode cloud-ingestion diagnostics, a concrete failure theory surfaced: configured tenant repositories can be private, and the orchestrator-side repo mirror may receive an access-denied response if the configured credentialRef is missing, stale, or lacks repository read permission.
This is already architecturally plausible and partially implemented:
GitMirror.cloneIfMissing() and GitMirror.fetch() inject credentials from credentialRef and throw stable KB_GITMIRROR_* codes on clone/fetch failures (ai/services/knowledge-base/helpers/gitMirror.mjs lines 428-474).
TenantRepoSyncService catches per-repo failures, but currently normalizes any non-KB_TENANT_REPO_SYNC_* code to KB_TENANT_REPO_SYNC_SYNC_FAILED before persisting per-repo outcome state (ai/daemons/orchestrator/services/TenantRepoSyncService.mjs lines 484-505).
DeploymentStateBridgeService relays lastErrorCode from that already-normalized per-repo outcome (ai/daemons/orchestrator/services/DeploymentStateBridgeService.mjs lines 715-720).
Live latest-open sweep: checked the latest 20 open issues at 2026-07-01T16:57:09Z; no equivalent found. Targeted GitHub searches for tenant-repo-sync GitMirror auth diagnostics credentialRef access denied KB_GITMIRROR returned no open issues. A2A in-flight sweep: checked the latest 30 messages; no competing lane claim for this provenance gap. Semantic KB sweep found the intended GitMirror and deployment-state surfaces, and also exposed the mismatch: KB/docs imply GitMirror failure codes are diagnostic inputs, while live code collapses them before the public snapshot.
The Problem
For a private upstream repository, a deployment can fail before ingestion for several different reasons:
credentialRef cannot resolve at runtime.
- the token resolves but lacks repository read permission;
- the repository URL is wrong, renamed, deleted, or unreachable;
- git/network transport fails independently of auth.
Today, operators can see a degraded per-repo state through tenantRepoSync diagnostics, but the structured public surface may only show KB_TENANT_REPO_SYNC_SYNC_FAILED. That is stable, but too broad: it forces the operator back to container logs to distinguish "token/access problem" from "repo missing" or "network blocked". In a cloud deployment, that weakens the diagnostic tools we just added for empty-KB incidents.
The important invariant is not "dump raw git stderr into MCP output". The invariant is "preserve enough non-secret provenance that an operator can branch correctly from the deployment diagnostic snapshot."
The Architectural Reality
The owning surfaces are already well separated:
TenantRepoAccessContract keeps cloneUrl credential-free and treats credentialRef as a reference, not secret material.
GitMirror owns low-level git clone/fetch/ref failure codes and redacts secret material before exposing errors.
TenantRepoSyncService owns per-repo aggregation and outcome state.
DeploymentStateBridgeService owns the bounded, public, graph-independent snapshot consumed by KB/MC diagnostics.
The fix belongs in the sync/diagnostic boundary, not in the scheduler. lastErrorCode can remain the stable tenant-sync umbrella code if compatibility needs it, but the lower-level redacted source code needs a bounded field that survives into the bridge snapshot.
The Fix
Preserve redacted underlying error provenance for per-repo tenant-repo-sync failures:
- In
TenantRepoSyncService, keep the existing stable outer lastErrorCode behavior where needed, but also persist a bounded source/subsystem code when the caught error carried one, e.g. lastSourceErrorCode, lastUnderlyingErrorCode, or another clearly named field.
- Preserve only non-secret categorical fields: source code, source family/subsystem, and optionally a bounded sanitized message class. Do not persist raw stderr, raw clone URLs, tokens, or raw repo identity.
- In
DeploymentStateBridgeService, relay the bounded provenance field through lastOutcome so inspect_deployment / get_deployment_state_snapshot can distinguish GitMirror clone/fetch/credential failures from generic ingest failures.
- Update docs and tests so the public diagnostic contract matches live behavior.
Contract Ledger Matrix
| Target Surface |
Source of Authority |
Proposed Behavior |
Fallback |
Docs |
Evidence |
TenantRepoSyncService per-repo failure state |
GitMirror error taxonomy + existing TenantRepoSyncErrors wrapper contract |
Preserve stable outer lastErrorCode and add a bounded underlying/source error code when available |
If no source code exists, field is null/omitted and lastErrorCode remains the operator branch point |
JSDoc in TenantRepoSyncService.mjs |
Unit test with KB_GITMIRROR_FETCH_FAILED and credential-ref failure |
DeploymentStateBridgeService tenantRepoSync.repos[].lastOutcome |
PR #14398 deployment diagnostics contract |
Include the bounded source/subsystem error field without exposing raw repo names, clone URLs, tokens, or stderr |
Snapshot remains backward-compatible with only lastErrorCode |
Cloud troubleshooting docs / KB MCP docs |
Unit test on snapshot projection |
| Cloud troubleshooting docs |
TenantIngestionModel.md, Troubleshooting.md, Day-0 tutorial |
Explain how to distinguish missing credential, clone/fetch/auth, and generic ingest failure from structured fields first, logs second |
Logs remain the fallback only for detailed redacted stderr |
learn/agentos/cloud-deployment/* |
Docs lint or focused static check |
Decision Record Impact
Aligned with ADR 0014. No ADR amendment expected: this does not change cloud lane classification or scheduler topology; it tightens the diagnostic contract for the existing pull-mode ingestion lane.
Acceptance Criteria
GitMirror clone/fetch failures with codes such as KB_GITMIRROR_CLONE_FAILED, KB_GITMIRROR_FETCH_FAILED, and KB_GITMIRROR_CREDENTIAL_REF_INVALID remain redacted and are preserved as bounded source provenance in per-repo tenant-repo-sync outcome state.
- Existing
lastErrorCode compatibility is defined explicitly: either it remains KB_TENANT_REPO_SYNC_SYNC_FAILED with a new source-code field, or a documented migration updates consumers safely.
DeploymentStateBridgeService exposes the bounded provenance through tenantRepoSync diagnostics while continuing to hash tenant/repo identity and exclude clone URLs, credential references, tokens, and raw stderr.
- Tests cover at least one clone/fetch failure, one unresolved credentialRef failure, and the bridge projection of the bounded source code.
- Docs update the pull-mode troubleshooting path so an operator can distinguish credential/access failures from generic ingest failures without first tailing container logs.
Out of Scope
- Changing how credentials are stored or injected.
- Embedding tokens in
cloneUrl.
- Adding live remote credential validation outside the actual git subprocess path.
- Changing scheduler classification or cadence; that is handled separately.
- Exposing raw git stderr in public MCP snapshots.
Avoided Traps
- Do not "fix" this by allowing credential-bearing clone URLs. The clean URL / reference-only credential boundary is correct.
- Do not add a one-off auth probe that duplicates
GitMirror; the mirror worker is already the authority for git transport behavior.
- Do not make remote diagnostics depend on container log access as the primary path. Logs are fallback detail, not the structured diagnostic contract.
Related
Related: #14396
Related: #14400
Handoff Retrieval Hints
Search terms: tenant-repo-sync GitMirror failure provenance, KB_GITMIRROR_FETCH_FAILED lastErrorCode, DeploymentStateBridgeService tenantRepoSync lastOutcome.
Origin Session ID: c0dfa949-22de-4daf-bbd2-1e093383fefc
Context
During pull-mode cloud-ingestion diagnostics, a concrete failure theory surfaced: configured tenant repositories can be private, and the orchestrator-side repo mirror may receive an access-denied response if the configured
credentialRefis missing, stale, or lacks repository read permission.This is already architecturally plausible and partially implemented:
GitMirror.cloneIfMissing()andGitMirror.fetch()inject credentials fromcredentialRefand throw stableKB_GITMIRROR_*codes on clone/fetch failures (ai/services/knowledge-base/helpers/gitMirror.mjslines 428-474).TenantRepoSyncServicecatches per-repo failures, but currently normalizes any non-KB_TENANT_REPO_SYNC_*code toKB_TENANT_REPO_SYNC_SYNC_FAILEDbefore persisting per-repo outcome state (ai/daemons/orchestrator/services/TenantRepoSyncService.mjslines 484-505).DeploymentStateBridgeServicerelayslastErrorCodefrom that already-normalized per-repo outcome (ai/daemons/orchestrator/services/DeploymentStateBridgeService.mjslines 715-720).Live latest-open sweep: checked the latest 20 open issues at 2026-07-01T16:57:09Z; no equivalent found. Targeted GitHub searches for
tenant-repo-sync GitMirror auth diagnostics credentialRef access denied KB_GITMIRRORreturned no open issues. A2A in-flight sweep: checked the latest 30 messages; no competing lane claim for this provenance gap. Semantic KB sweep found the intended GitMirror and deployment-state surfaces, and also exposed the mismatch: KB/docs imply GitMirror failure codes are diagnostic inputs, while live code collapses them before the public snapshot.The Problem
For a private upstream repository, a deployment can fail before ingestion for several different reasons:
credentialRefcannot resolve at runtime.Today, operators can see a degraded per-repo state through
tenantRepoSyncdiagnostics, but the structured public surface may only showKB_TENANT_REPO_SYNC_SYNC_FAILED. That is stable, but too broad: it forces the operator back to container logs to distinguish "token/access problem" from "repo missing" or "network blocked". In a cloud deployment, that weakens the diagnostic tools we just added for empty-KB incidents.The important invariant is not "dump raw git stderr into MCP output". The invariant is "preserve enough non-secret provenance that an operator can branch correctly from the deployment diagnostic snapshot."
The Architectural Reality
The owning surfaces are already well separated:
TenantRepoAccessContractkeepscloneUrlcredential-free and treatscredentialRefas a reference, not secret material.GitMirrorowns low-level git clone/fetch/ref failure codes and redacts secret material before exposing errors.TenantRepoSyncServiceowns per-repo aggregation and outcome state.DeploymentStateBridgeServiceowns the bounded, public, graph-independent snapshot consumed by KB/MC diagnostics.The fix belongs in the sync/diagnostic boundary, not in the scheduler.
lastErrorCodecan remain the stable tenant-sync umbrella code if compatibility needs it, but the lower-level redacted source code needs a bounded field that survives into the bridge snapshot.The Fix
Preserve redacted underlying error provenance for per-repo tenant-repo-sync failures:
TenantRepoSyncService, keep the existing stable outerlastErrorCodebehavior where needed, but also persist a bounded source/subsystem code when the caught error carried one, e.g.lastSourceErrorCode,lastUnderlyingErrorCode, or another clearly named field.DeploymentStateBridgeService, relay the bounded provenance field throughlastOutcomesoinspect_deployment/get_deployment_state_snapshotcan distinguish GitMirror clone/fetch/credential failures from generic ingest failures.Contract Ledger Matrix
TenantRepoSyncServiceper-repo failure stateGitMirrorerror taxonomy + existingTenantRepoSyncErrorswrapper contractlastErrorCodeand add a bounded underlying/source error code when availablenull/omitted andlastErrorCoderemains the operator branch pointTenantRepoSyncService.mjsKB_GITMIRROR_FETCH_FAILEDand credential-ref failureDeploymentStateBridgeServicetenantRepoSync.repos[].lastOutcome#14398deployment diagnostics contractlastErrorCodeTenantIngestionModel.md,Troubleshooting.md, Day-0 tutoriallearn/agentos/cloud-deployment/*Decision Record Impact
Aligned with ADR 0014. No ADR amendment expected: this does not change cloud lane classification or scheduler topology; it tightens the diagnostic contract for the existing pull-mode ingestion lane.
Acceptance Criteria
GitMirrorclone/fetch failures with codes such asKB_GITMIRROR_CLONE_FAILED,KB_GITMIRROR_FETCH_FAILED, andKB_GITMIRROR_CREDENTIAL_REF_INVALIDremain redacted and are preserved as bounded source provenance in per-repo tenant-repo-sync outcome state.lastErrorCodecompatibility is defined explicitly: either it remainsKB_TENANT_REPO_SYNC_SYNC_FAILEDwith a new source-code field, or a documented migration updates consumers safely.DeploymentStateBridgeServiceexposes the bounded provenance throughtenantRepoSyncdiagnostics while continuing to hash tenant/repo identity and exclude clone URLs, credential references, tokens, and raw stderr.Out of Scope
cloneUrl.Avoided Traps
GitMirror; the mirror worker is already the authority for git transport behavior.Related
Related: #14396 Related: #14400
Handoff Retrieval Hints
Search terms:
tenant-repo-sync GitMirror failure provenance,KB_GITMIRROR_FETCH_FAILED lastErrorCode,DeploymentStateBridgeService tenantRepoSync lastOutcome.Origin Session ID: c0dfa949-22de-4daf-bbd2-1e093383fefc