TenantRepoSyncService class |
Epic #11731 + ADR 0014 tenant-repo pull-ingestion amendment |
extends Base, singleton with reactive configs for enablement, concurrency limit, and cadence/backoff policy; exposes syncAllTenantRepos() and syncOneTenantRepo({tenantId, repoSlug}). |
Disabled service -> all syncs no-op with operator-visible status; service init failure -> lane records a stable failure and skips the tick without taking down the Orchestrator. |
JSDoc + TenantIngestionModel.md reference |
Unit tests with stubbed GitMirror + envelope builder: per-repo iteration and disabled/init-failure gates |
tenant-repo-sync Orchestrator lane |
ADR 0014 amendment + #11740 lane classification |
Add a serviceTask entry in TaskDefinitions.mjs, classify it as cloud-deployable, and wire a deployment-aware enable toggle. Cloud profile defaults enabled when tenant repos exist; local Neo-maintainer profile defaults disabled unless explicitly enabled. |
Env override disables the lane; missing tenant-repo config makes the tick a no-op, not an error. |
TaskDefinitions.mjs / Orchestrator JSDoc + cloud deployment docs |
Unit test: task definition exists; Orchestrator getter/toggle chooses the correct deployment default and env override |
Per-repo iteration (syncAllTenantRepos) |
#11790 AC1 + #11787 TenantRepoAccessContract config |
Iterate KnowledgeBaseIngestionService.getTenantConfig().tenantRepos[]; for each entry: clone-if-missing, fetch, build bootstrap/incremental envelope, and ingest. Per-repo failures are isolated. |
Empty tenantRepos[] -> no-op; one repo failure logs/returns a per-repo error while remaining repos continue; failed repo revision state remains unchanged. |
JSDoc |
Unit test: three repo configs, middle fetch fails, other two complete, failed repo state unchanged |
| Manual operator run path |
#11790 AC3 |
Provide a manual operator surface (CLI or MCP op) to refresh one configured repo or all repos. CLI path should follow existing commander-backed script patterns if selected. |
Missing required selector -> parser/schema rejects; unknown tenant or repo -> stable KB_TENANT_REPO_SYNC_* error; no configured repos -> no-op with visible log/summary. |
TenantIngestionModel.md operator-run section |
Unit test for parser/schema accept/reject; dispatch test stubbing TenantRepoSyncService |
| Per-repo cadence + jitter/backoff |
#11790 AC2 + Orchestrator scheduling pure-function pattern |
Pure scheduling function computes next-due state from last sync attempt, configured cadence, deterministic repoSlug/tenant salt jitter, and failure backoff. |
First bootstrap wave is jittered; repeated failure backs off without starving healthy repos; success resets backoff. |
JSDoc on scheduling function |
Unit test: jitter deterministic per repoSlug/tenant; backoff grows on consecutive failures and resets on success |
Revision-state persistence (lastIngestedRev writeback) |
#11789 envelope-builder handoff contract |
After a successful ingestSourceFiles() summary, service persists lastIngestedRev = envelope.headRevision for that {tenantId, repoSlug} through the selected tenant-sync state surface. |
Persistence failure leaves revision unchanged and returns/logs KB_TENANT_REPO_SYNC_MANIFEST_UPDATE_FAILED; next tick re-detects the same diff and retries idempotently. |
JSDoc on state handoff |
Unit test: successful sync advances revision; persistence failure leaves revision unchanged and next run sees same diff |
| Concurrency-limit gate |
#11790 implementation discipline |
Reactive concurrencyLimit config caps simultaneous git/ingest work to avoid multi-tenant CPU/network exhaustion. |
concurrencyLimit: 1 serializes; invalid or omitted limit falls back to a conservative default. |
JSDoc |
Unit test: five repos with limit 2 never exceed two in-flight syncs |
Cross-lane interaction with kbSync |
ADR 0014 tenant-repo amendment |
tenant-repo-sync is separate from local-only kbSync; no repointing, shared lock, or shared local-checkout assumption. |
If both lanes write to the KB collection, existing tenant/repo scoping prevents cross-tenant leakage; lane state remains independent. |
TenantIngestionModel.md operator note + ADR 0014 anti-pattern reinforcement |
Unit/concurrency test: both lanes' state independent; no cross-pollution of revision or manifest state |
| Stable error codes |
Cross-sub consistency |
KB_TENANT_REPO_SYNC_* prefix for service/scheduler/manual-run errors, e.g. TENANT_NOT_FOUND, REPO_NOT_CONFIGURED, SYNC_FAILED, MANIFEST_UPDATE_FAILED, CONCURRENCY_GATE_TIMEOUT; callers branch on code, not message prose. |
All surfaced errors use the #11787/#11788 redaction path for credential-bearing hints or git stderr before logging/returning. |
JSDoc on error codes |
Unit test: each error path returns the expected stable code and redacts injected fake secrets |
Context
Sub 5 of Epic #11731 (Server-side tenant-repo ingestion for cloud Agent OS deployments), graduated from Discussion #11782. This sub is the trigger layer — the
tenant-repo-synclane that drives sub 3 (GitMirror) + sub 4 (envelope builder).The Problem
The persistent-mirror refresh cycle needs a driver. Discussion #11782 OQ2 resolved: periodic refresh + a manual run path first; webhook is a later accelerator (a webhook-first design reintroduces per-repo tenant wiring and is explicitly deferred).
The Architectural Reality
A new cloud-deployable
tenant-repo-syncOrchestratorscheduler lane +TenantRepoSyncService— distinct from the local-onlyprimary-dev-synclane. ADR 0014's §2.1 lane taxonomy is amended (sub 1 / #11740) to classifytenant-repo-syncas cloud-deployable. File placement:structural-pre-flightat implementation — candidateai/daemons/services/TenantRepoSyncService.mjs(sibling ofPrimaryRepoSyncService.mjs), wired intoTaskDefinitions.mjsas aserviceTasklane.The Fix
TenantRepoSyncServiceconsumingGitMirror(sub 3) + the envelope builder (sub 4): iterate configured tenant repos → clone-if-missing / fetch → diff → ingest.tenant-repo-syncperiodicOrchestratorlane: per-repo cadence with jitter/backoff.tenant-repo-sync= cloud-deployable (handed to sub 1 / #11740).Acceptance Criteria
TenantRepoSyncServiceiterates configured tenant repos through the sub-3 / sub-4 pipeline.tenant-repo-syncOrchestrator lane with per-repo cadence + jitter/backoff.structural-pre-flightapplied to new.mjsplacements.Out of Scope
GitMirror(sub 3) and the envelope builder (sub 4).Contract Ledger
Per the Contract Completeness Gate for tickets introducing public or consumed surfaces. #11790 introduces the
tenant-repo-syncOrchestrator scheduler lane andTenantRepoSyncService, consuming sub 2 (TenantRepoAccessContract, #11787), sub 3 (GitMirror, #11788), sub 4 (envelope builder, #11789), and sub 1's cloud-deployable lane classification (#11740 / ADR 0014 amendment). Contract surface is enumerated below.TenantRepoSyncServiceclassextends Base, singleton with reactive configs for enablement, concurrency limit, and cadence/backoff policy; exposessyncAllTenantRepos()andsyncOneTenantRepo({tenantId, repoSlug}).TenantIngestionModel.mdreferencetenant-repo-syncOrchestrator laneserviceTaskentry inTaskDefinitions.mjs, classify it as cloud-deployable, and wire a deployment-aware enable toggle. Cloud profile defaults enabled when tenant repos exist; local Neo-maintainer profile defaults disabled unless explicitly enabled.TaskDefinitions.mjs/OrchestratorJSDoc + cloud deployment docssyncAllTenantRepos)KnowledgeBaseIngestionService.getTenantConfig().tenantRepos[]; for each entry: clone-if-missing, fetch, build bootstrap/incremental envelope, and ingest. Per-repo failures are isolated.tenantRepos[]-> no-op; one repo failure logs/returns a per-repo error while remaining repos continue; failed repo revision state remains unchanged.commander-backed script patterns if selected.KB_TENANT_REPO_SYNC_*error; no configured repos -> no-op with visible log/summary.TenantIngestionModel.mdoperator-run sectionTenantRepoSyncServicelastIngestedRevwriteback)ingestSourceFiles()summary, service persistslastIngestedRev = envelope.headRevisionfor that{tenantId, repoSlug}through the selected tenant-sync state surface.KB_TENANT_REPO_SYNC_MANIFEST_UPDATE_FAILED; next tick re-detects the same diff and retries idempotently.concurrencyLimitconfig caps simultaneous git/ingest work to avoid multi-tenant CPU/network exhaustion.concurrencyLimit: 1serializes; invalid or omitted limit falls back to a conservative default.kbSynctenant-repo-syncis separate from local-onlykbSync; no repointing, shared lock, or shared local-checkout assumption.TenantIngestionModel.mdoperator note + ADR 0014 anti-pattern reinforcementKB_TENANT_REPO_SYNC_*prefix for service/scheduler/manual-run errors, e.g.TENANT_NOT_FOUND,REPO_NOT_CONFIGURED,SYNC_FAILED,MANIFEST_UPDATE_FAILED,CONCURRENCY_GATE_TIMEOUT; callers branch oncode, not message prose.Behavioral invariants:
Out of contract scope:
Related
Origin Session ID
39185c66-a107-46ea-b0bf-eb4fa1137257