LearnNewsExamplesServices
Frontmatter
id11788
titleGitMirror primitive + persistent mirror acquisition service
stateClosed
labels
enhancementaiarchitecture
assigneesneo-gpt
createdAtMay 22, 2026, 11:35 PM
updatedAtJun 7, 2026, 7:14 PM
githubUrlhttps://github.com/neomjs/neo/issues/11788
authorneo-opus-ada
commentsCount2
parentIssue11731
subIssues[]
subIssuesCompleted0
subIssuesTotal0
blockedBy[]
blocking[x] 11789 Diff-to-ingest envelope builder for tenant-repo sync
closedAtMay 24, 2026, 12:51 PM

GitMirror primitive + persistent mirror acquisition service

Closed v13.0.0/archive-v13-0-0-chunk-13 enhancementaiarchitecture
neo-opus-ada
neo-opus-ada commented on May 22, 2026, 11:35 PM

Context

Sub 3 of Epic #11731 (Server-side tenant-repo ingestion for cloud Agent OS deployments), graduated from Discussion #11782. This sub builds the GitMirror primitive + the persistent mirror acquisition layer — the structural core of the design (Discussion #11782 OQ1 → Option A, with GitMirror as the disciplined form of Option C).

The Problem

The deployment must keep tenant repos as persistent mirrors — clone-once into a container volume, then fetch for updates — rather than re-cloning each refresh (wasteful) or re-pointing the local-only kbSync/primary-dev-sync lanes at tenant content (ADR 0014 §5.2 anti-pattern).

The Architectural Reality

PrimaryRepoSyncService does NOT clone and is dev-hardcoded + local-only — it is not reused or rewritten. GitMirror is a new lower-level primitive; TenantRepoSyncService (sub 5's lane consumes it). File placement validated via structural-pre-flight at implementation — candidate: ai/daemons/services/ (sibling of PrimaryRepoSyncService.mjs, DreamService.mjs); the GitMirror primitive's exact home gets its own structural-pre-flight pass.

The Fix

  • A GitMirror primitive owning: clone-if-missing (via the credentialed reference from sub 2), fetch, resolve-head, ancestor-check, and changed+deleted file diff between two revisions.
  • Per-{tenantId, repoSlug} mirror paths, computed from the clean normalized identity only (never from a credentialed URL).
  • A persistent repo-mirror volume in the deployment compose (alongside ADR 0014 §2.2's chroma-data / shared-sqlite-data); redeploy-survival.
  • Transient credential injection: HTTPS via GIT_ASKPASS/credential.helper resolving sub 2's credentialRef; SSH via GIT_SSH_COMMAND + the referenced deploy-key. No token in URL or argv.

Acceptance Criteria

  • GitMirror primitive: clone-if-missing / fetch / resolve-head / ancestor-check / revision-diff.
  • Per-{tenantId, repoSlug} mirror paths, derived from clean identity only.
  • Persistent repo-mirror volume in the deployment compose; survives redeploy.
  • Transient credential injection (no secret in URL/argv); passes sub 2's no-leak test classes.
  • structural-pre-flight applied to all new .mjs file placements.

Out of Scope

  • The config + contract surface (sub 2 / #11731).
  • Turning the diff into an ingestSourceFiles() envelope (sub 4 / #11731).
  • The scheduler lane (sub 5 / #11731).

Contract Ledger

Per the Contract Completeness Gate for public/consumed surfaces, #11788 introduces the canonical GitMirror primitive consumed by the tenant-repo-sync scheduler lane (#11790), the diff-to-ingest envelope builder (#11789), and the deployment volume layer (ADR 0014). Downstream consumers MUST V-B-A against this matrix.

Target Surface Source of Authority Proposed Behavior Fallback / Edge Case Docs Evidence
GitMirror.cloneIfMissing({tenantId, repoSlug, cloneUrl, credentialRef}) #11788 AC1; #11731 AC3; #11787 credentialed repo-access contract; ADR 0014 tenant-repo-sync amendment Clone via injected credential only when the mirror path is absent; existing clean mirror is idempotent no-op. Git subprocess failure throws KB_GITMIRROR_CLONE_FAILED with stderr redacted through redactTenantRepoSecrets; invalid mirror path throws KB_GITMIRROR_MIRROR_PATH_INVALID. JSDoc on GitMirror; TenantIngestionModel.md operator note if behavior is user-visible. Unit coverage using local fixture repos for clone/no-op; no-secret assertions for argv/log/output.
GitMirror.fetch({tenantId, repoSlug, credentialRef}) #11788 AC1; #11731 persistent mirror model; ADR 0014 tenant-repo-sync lane Run git fetch --all --prune inside an existing mirror and return {fetchedAt, newRevisions} or equivalent implementation payload. Missing mirror is not silently cloned unless caller explicitly invoked clone path; git failure throws KB_GITMIRROR_FETCH_FAILED with redacted stderr. JSDoc on GitMirror. Unit coverage for successful fetch, missing mirror behavior, and redacted failure text.
GitMirror.resolveHead({tenantId, repoSlug, ref}) #11788 AC1; #11731 update model Resolve branch, tag, or SHA to a full SHA in the mirror. Unknown ref throws KB_GITMIRROR_REF_NOT_FOUND; callers branch on .code, not .message. JSDoc on GitMirror. Unit coverage for branch/tag/SHA resolution and missing-ref failure.
GitMirror.isAncestor({tenantId, repoSlug, ancestor, descendant}) #11788 AC1; #11731 AC4 force-push fallback model Return true iff ancestor is reachable from descendant, enabling fast-forward safety and skip-no-op detection. Git failure throws stable KB_GITMIRROR_* error with redacted stderr; false is reserved for valid non-ancestor results. JSDoc on GitMirror. Unit coverage for ancestor, non-ancestor, and failure cases.
GitMirror.diffRevisions({tenantId, repoSlug, baseRevision, headRevision}) #11788 AC1; #11789 envelope-builder dependency; #11731 AC4 Return repo-relative {addedOrChanged: String[], deleted: String[]} paths between two revisions. Git failure throws KB_GITMIRROR_DIFF_FAILED; invalid paths are never emitted outside the mirror root. JSDoc on GitMirror; #11789 owns envelope-shape docs. Unit coverage for added, changed, deleted, and empty diffs using local fixture repos.
Mirror path layout via deriveTenantRepoMirrorPath({mirrorRoot, tenantId, repoSlug}) #11787 shipped contract; #11788 AC2; ADR 0014 volume persistence Map {tenantId, repoSlug} to <mirrorRoot>/tenant-repos/<tenantId>/<repoSlugSegments> using clean, path-traversal-safe segments; credential-free by construction. Invalid tenant/repo identity throws KB_GITMIRROR_MIRROR_PATH_INVALID or existing path-validation error; direct consumer path manipulation is forbidden. TenantIngestionModel.md; JSDoc remains with the helper. Path derivation tests proving traversal rejection and no secret material in paths.
Credential injection - HTTPS #11787 credentialed repo-access contract; #11788 AC4 Use GIT_ASKPASS plus scoped git credential configuration to resolve credentialRef per git invocation; never persist secret in URL, argv, graph config, or instance state. Missing or invalid credential reference fails before durable logging; stderr is redacted before logging/telemetry. TenantIngestionModel.md; GitMirror JSDoc. No-leak tests with fake secrets absent from argv/log/output/persisted state.
Credential injection - SSH #11787 credentialed repo-access contract; #11788 AC4 Use GIT_SSH_COMMAND referencing the deploy-key path resolved from credentialRef; never place key material in URL or argv. Missing deploy key / invalid reference fails with stable redacted error; no key material in logs. TenantIngestionModel.md; GitMirror JSDoc. No-leak tests with fake key path/reference behavior and redacted failures.
Persistent repo-mirror volume #11788 AC3; ADR 0014 volume topology; ai/deploy/docker-compose.yml existing volume pattern Add a deployment compose volume mounted at mirrorRoot, sibling to chroma-data and shared-sqlite-data; mirror contents survive container redeploy. Local/unit tests may use temp dirs; production compose owns persistent volume wiring. Deployment compose docs / cloud deployment guide update. Compose/static assertion or documented verification showing the named volume and mount path.
Logged stderr / telemetry #11787 redaction contract; #11788 AC4; #11791 telemetry follow-up All captured git stdout/stderr and thrown errors pass through redactTenantRepoSecrets({secretHints}) before durable logging or telemetry emission. Redaction is a backstop for git error text; stable error .code remains branchable after redaction. GitMirror JSDoc; #11791 owns broader operator telemetry docs. No-secret tests proving fake tokens are absent from all durable/error surfaces.
Stable error codes #11788 API contract; downstream #11789/#11790 consumers Use KB_GITMIRROR_* codes: CLONE_FAILED, FETCH_FAILED, REF_NOT_FOUND, DIFF_FAILED, MIRROR_PATH_INVALID, CREDENTIAL_REF_INVALID unless implementation discovers a narrower code list and updates this ledger before PR open. Consumers branch on .code; .message stays human-readable and redactable. JSDoc on errors thrown by GitMirror. Unit coverage asserting error codes for representative failure modes.

Behavioral invariants (cross-contract):

  • Mirror path derivation is the only public surface that maps {tenantId, repoSlug} to filesystem path; consumers MUST go through deriveTenantRepoMirrorPath from #11787.
  • Credential resolution happens at git-invocation time and is never cached in GitMirror instance state; credentialRef is the only durable handle.
  • Git subprocess calls use explicit {env, cwd} overrides and do not rely on inherited shell state for credentials.
  • Mirror path layout is stable across GitMirror instance recreations and container redeploys; persistence is the volume contract, not in-memory state.

Out of contract scope (covered by sibling subs):

  • Tenant config persistence / no-secret boundary → sub 2 #11787 (shipped by PR #11880).
  • Diff to ingestSourceFiles() envelope shape → sub 4 #11789.
  • Scheduler lane cadence + task definition → sub 5 #11790.
  • Credential acquisition / rotation / vault integration → deployment-operator responsibility.

Related

  • Parent epic: #11731.
  • Origin Discussion: #11782 (OQ1, OQ3, OQ7).
  • ADR: 0014 (deployment topology / volumes).

Origin Session ID

39185c66-a107-46ea-b0bf-eb4fa1137257

tobiu referenced in commit 09a3d60 - "feat(kb): add tenant repo git mirror (#11788) (#11881) on May 24, 2026, 12:51 PM
tobiu closed this issue on May 24, 2026, 12:51 PM
tobiu referenced in commit 41a93c4 - "feat(orchestrator): cloud-deployable tenant-repo-sync scheduler lane (#11790) (#11940) on May 25, 2026, 7:48 AM
tobiu referenced in commit 1bfee42 - "docs(day0): rewrite Milestone 6 to first-class pull mode (#12052) (#12053) on May 27, 2026, 12:58 AM