AzCopy – Transfer data with re-startable mode and SAS Token

  • Article

Recently, we released a new version of AzCopy with a set of new features.

You can always download latest version from aka.ms/AzCopy.

New features added

  • /DestSAS and /SourceSAS: This option allows access to storage containers and blobs with a SAS (Shared Access Signature) token. SAS token, which is generated by the storage account owner, grants access to specific containers and blobs with specifc permissions and for a specified period of time. Permissions include LIST, READ, WRITE or DELETE.
    Currently, we support SAS token while you upload data to Azure Storage or download data from Azure Storage. For blob copy, we support SAS Token for the source location.
    Please refer to this blog post for more information on How to create a Shared Access Signature.
  • Enhancement of re-startable mode: In previous version, the re-startable mode is supported for interruption caused by network or other issues during file transfer but it only supports restarting the transfer from the beginning of the interrupted file. Imagine if the interrupted file is a large file (e.g. a VHD file) and most part of it has already been transferred to Azure Storage, it would be time-consuming to restart the transfer of entire file again. So to improve this scenario, we enhanced re-startable mode and made it restart the transfer from the point of interruption. For block blob, we chose 4Mb as the size of data chunk.

 

Example1: Upload all files from a local directory to a container using SAS token which offers permits for list and write

AzCopy C:\blobData https://xyzaccount.blob.core.windows.net/xyzcontainer /DestSAS:”?sr=c&si=mypolicy&sig=XXXXX” /s

/DestSAS here is for you to specify the SAS token to access storage container, it should be enclosed in quotes.

 

Example2: Download all blobs from a container to a local directory and then delete them from source (by using /Mov). The SAS token offers list, read and delete permission.

AzCopy https://xyzaccount.blob.core.windows.net/xyzcontainer C:\blobData /SourceSAS:”?sr=c&si=mypolicy&sig=XXXXX” /MOV /s

 

Example3: Upload all files from a local directory to a container in re-startable mode.

AzCopy C:\blobData https://myaccount.blob.core.windows.net/mycontainer /destkey:key /Z:restart.log /s

/Z is the parameter to turn on the re-startable mode. ‘restart.log’ is the customer-defined name of the journal file which will be saved in local.

For instance, if “C:\blobData” folder contains four files below and the size of file2.vhd is 1GB.

C:\blobData\file1.docx

C:\blobData\file2.vhd

C:\blobData\file3.txt

C:\blobData\file4.txt

Assume an interrupt occurs while copying the “file2.vhd”, and 90% of the file has been transferred to Azure already. Then upon restarting, we will only transfer the remaining 10% of “file2.vhd” and another two remaining files.

 

Jason Tang