我正在寻找通过Azure存储帐户中的两个文件共享文件同步文件的最佳方式。 现在看起来像AZCopy是最好的选择,但在特定案例或过滤器的情况下,它将无法正常工作。Azure文件共享同步
例如:
- 我想复制某个日期(上周,一个月 等)之后添加的文件。
- 跳过一些文件夹/子文件夹
使用PowerShell也选择,但它可能是慢同步的文件数量巨大。
我有一些脚本或包装在AZCopy之上,可以帮助涵盖文件夹和文件类型的过滤吗?
感谢 伊霍尔
我正在寻找通过Azure存储帐户中的两个文件共享文件同步文件的最佳方式。 现在看起来像AZCopy是最好的选择,但在特定案例或过滤器的情况下,它将无法正常工作。Azure文件共享同步
例如:
使用PowerShell也选择,但它可能是慢同步的文件数量巨大。
我有一些脚本或包装在AZCopy之上,可以帮助涵盖文件夹和文件类型的过滤吗?
感谢 伊霍尔
这里是同步的一些文件夹中的脚本:
function Copy-AzureFileStorageWithPS{
<#
.SYNOPSIS
Copy a file share from a from one storage account to another using PowerShell
.DESCRIPTION
This function will copy a file share or specific folder inside of file share from one storage account to another
.PARAMETER SourceStorageAccountName
.PARAMETER DestStorageAccountName
.PARAMETER SourceResourceGroupName
.PARAMETER DestResourceGroupName
.PARAMETER FoldersToSkip
.PARAMETER FoldersToCopy
.EXAMPLE
. .\Copy-AzureFileStorageWithPS -SourceStorageAccountName "some" -DestStorageAccountName "other" -SourceResourceGroupName "RG" -DestResourceGroupName "OtherRg"
#>
[CmdletBinding()]
param(
[Parameter (Mandatory = $true)]
[string] $SourceStorageAccountName,
[Parameter (Mandatory = $true)]
[string] $DestStorageAccountName,
[Parameter (Mandatory = $true)]
[string] $SourceResourceGroupName,
[Parameter (Mandatory = $true)]
[string] $DestResourceGroupName,
[Parameter (Mandatory = $true)]
[string] $FoldersToSkip = "",
[Parameter (Mandatory = $true)]
[string] $FoldersToCopy = ""
)
$exclusions = $FoldersToSkip.ToLower().Split(',').Trim()
$copyList = $FoldersToCopy.ToLower().Split(',').Trim()
$SourceStorageAccountKey = (Get-AzureRmStorageAccountKey -Name $SourceStorageAccountName -ResourceGroupName $SourceResourceGroupName)[0].Value
$DestStorageAccountKey = (Get-AzureRmStorageAccountKey -Name $DestStorageAccountName -ResourceGroupName $DestResourceGroupName)[0].Value
$SourceContext = New-AzureStorageContext -StorageAccountName $SourceStorageAccountName -StorageAccountKey $SourceStorageAccountKey
$DestContext = New-AzureStorageContext -StorageAccountName $DestStorageAccountName -StorageAccountKey $DestStorageAccountKey
#Get all shares from Source SA
$shares = Get-AzureStorageshare -Context $SourceContext
foreach($share in $shares){
$destShare = Get-AzureStorageshare -Context $DestContext -Name $share.Name -ErrorAction SilentlyContinue
if(-Not $destShare)
{
$destShare = New-AzureStorageShare $share.Name -Context $DestContext
Write-Host "Successfully created share $($share.Name) in Account $($DestContext.StorageAccountName)"
}
$shareContent = Get-AzureStorageFile -Share $share
foreach($content in $shareContent)
{
if(-Not $exclusions.Contains($content.Name.ToLower())){
if($copyList.Length -ne 0){
if($copyList.Contains($content.Name.ToLower())){
Write-Output "Starting copy of $($content.Name) from $($content.Uri.AbsoluteUri) to $($destShare.Uri.AbsoluteUri+"/"+$content.Name)"
Copy-AonAzureFolders -SourceShare $share -SourceFolder $content -DestShare $destShare -SourceContext $SourceContext -DestContext $DestContext
}
}
else
{
Write-Output "Starting copy of $($content.Name) from $($content.Uri.AbsoluteUri) to $($destShare.Uri.AbsoluteUri+"/"+$content.Name)"
Copy-AonAzureFolders -SourceShare $share -SourceFolder $content -DestShare $destShare -SourceContext $SourceContext -DestContext $DestContext
}
}
else{
Write-Output "Excluding folder $($content.Name) from copying to $($destShare.Name)"
}
}
}
}
function Copy-AonAzureFolders
{
[CmdletBinding()]
param(
[Parameter (Mandatory = $true)]
[Microsoft.WindowsAzure.Storage.File.CloudFileShare] $SourceShare,
[Parameter (Mandatory = $true)]
[Microsoft.WindowsAzure.Storage.File.CloudFileDirectory] $SourceFolder,
[Parameter (Mandatory = $true)]
[Microsoft.WindowsAzure.Storage.File.CloudFileShare] $DestShare,
[Parameter (Mandatory = $true)]
[Microsoft.WindowsAzure.Commands.Common.Storage.AzureStorageContext] $SourceContext,
[Parameter (Mandatory = $true)]
[Microsoft.WindowsAzure.Commands.Common.Storage.AzureStorageContext] $DestContext
)
$pathWithoutShare = $($SourceFolder.Uri.LocalPath).Replace("/$($SourceShare.Name)","")
$shareFolders = Get-AzureStorageFile -Share $SourceShare -Path $pathWithoutShare | Get-AzureStorageFile | Where-Object {$_.GetType().Name -eq "CloudFileDirectory"}
$folderExist = Get-AzureStorageFile -Share $DestShare -Path $pathWithoutShare -ErrorAction SilentlyContinue
if(-Not $folderExist){
$newFolder = New-AzureStorageDirectory -Share $DestShare -Path $pathWithoutShare
}
foreach($folder in $shareFolders)
{
#Recursive call
Copy-AonAzureFolders -SourceShare $SourceShare -SourceFolder $folder -DestShare $DestShare -SourceContext $SourceContext -DestContext $DestContext
}
$shareFiles = Get-AzureStorageFile -Share $SourceShare -Path $($pathWithoutShare+"/") | Get-AzureStorageFile | Where-Object {$_.GetType().Name -eq "CloudFile"}
foreach($file in $shareFiles)
{
$fullFilePath = $pathWithoutShare + "/" + $($file.Name)
Write-Host "-SrcShareName $($SourceShare.Name) -SrcFilePath $fullFilePath -DestShareName $($DestShare.Name) -DestFilePath $fullFilePath"
# copy a file to the new directory
$copyfile = Start-AzureStorageFileCopy -SrcShareName $($SourceShare.Name) -SrcFilePath $fullFilePath `
-DestShareName $($DestShare.Name) -DestFilePath $fullFilePath -Context $SourceContext -DestContext $DestContext -Force
$status = $copyfile | Get-AzureStorageFileCopyState
while ($status.Status -eq "Pending")
{
$status = $copyfile | Get-AzureStorageFileCopyState
Start-Sleep -Milliseconds 150
}
$copyfile
}
}
希望能够帮助您 但它是相当缓慢
据我所知,目前还没有办法保住两艘Azure的文件共享湛蓝本身同步。
作为一种解决方法,您需要在Azure上构建一个虚拟机并在其上安装这两个共享。然后你只需要找到一个可以保持两个文件夹同步的解决方案。在Windows和Linux上都有几种解决方案。
的工具用Python编写的,但可以使用泊坞推出:
https://github.com/dokkur/azure-fileshare-sync
运行:
AZURE_STORAGE_ACCOUNT=account\
AZURE_STORAGE_ACCESS_KEY=key\
python -m src.azure_sync ./local/path sharename[/remote/path]
您能否通过提供详细信息来更新您的问题?但是如果出现特定情况或过滤器将无法工作? –
文件同步与文件复制不同。 AzCopy不会提供这种类型的同步功能。最终,您选择的用于文件同步的工具取决于您 - 工具推荐问题与主题无关(并且考虑到所有用例需要考虑,仅仅掀起文件同步脚本并不那么简单)。 –
什么推动了同步的场景?似乎有平台功能可以解决问题... –