AzCopy – Using Cross Account Copy Blob

  • Article

You can always download latest version from aka.ms/AzCopy.

New features have been added in this release

  • Support of Cross-Account Copy Blob: AzCopy allows you to copy blobs within same storage accounts or between different storage accounts (visit this blog post for more details on cross account blob copy). This enables you move blobs from one account to another efficiently with respect to cost and time. The data transfer is done by the storage service, thus eliminating the need for you to download each blob from the source and then upload to the destination. You can also use /Z to execute the blob copy in re-startable mode.

  • Added /MOV: This option allows you move files and delete them from source after copying. Assuming you have delete permissions on the source, this option applies regardless of whether the source is Windows Azure Storage or the local file system.

  • Added /NC: This option allows you to specify the concurrent network calls. By default, when you upload files from local computer to Windows Azure Storage, AzCopy will initiate network calls up to eight times the number of cores this local computer had to execute concurrent tasks. For example, if your local computer has four cores, then AzCopy will initiate up to 32 (eight times of 4) network calls at one time. However, if you want to limit the concurrency to throttle local CPU and bandwidth usage, you can specify the maximum concurrent network calls by using /NC. The value specified here is the absolute count and will not be multiplied by the core count. So in the above example, to reduce the concurrent network calls by half, you would specify /NC:16

  • Added /SNAPSHOT: This option allows you transfer blob with snapshots. This is a semantic change, as in AzCopy CTP 1 (Released in October, 2012), it would transfer a blob’s snapshots by default.  However starting from this version, by default AzCopy won’t transfer any snapshots while copying a blob. Only with /SNAPSHOT specified, AzCopy will actually transfer all the snapshots of a blob to destination, however these snapshots will became separate blobs instead of snapshots of original base blob in destination, so each of the blobs will be full charged for (not block reuse between them). The transferred blob snapshots will be renamed in this format: [blob-name] (snapshot-time)[extension].

    For example if readme.txt is a source blob and there are 3 snapshots created for it in source container, then after using /SNAPSHOT there will be 3 more separate blobs created in destination container where their names will looks like

    readme (2013-02-25 080757).txt
    readme (2012-12-23 120657).txt
    readme (2012-09-12 090521).txt

    For the billing impact compare blob snapshots to separate blob, please refer to this blog post Understanding Windows Azure Storage Billing

  • Added /@:response-file: Thisallows you store parameters in a file and they will be processed by AzCopy just as if they had been specified on the command line. Parameters in the response file can be divided as several lines but each single parameter must be in one line (breaking 1 parameter into 2 lines is not supported). AzCopy parses each line as if it is a single command line into a list of parameters and concatenates all the parameter lists into one list, which is treated by AzCopy as from a single command line. Multiple response files can be specified, but nested response file is not supported, instead it will be parsed as a location parameter or file pattern. Escape characters are not supported, except that “” in a quoted string will be parsed as a single quotation mark. Note that /@: after /- does not means a response file, it is treated as a location parameter or file pattern like other parameters

Examples

Here are some examples that illustrate the new features in this release.

Copy all blobs from one container to another container under different storage account

AzCopy https://<sourceaccount>.blob.core.windows.net/<sourcecontainer>/ https://<destaccount>.blob.core.windows.net/<destcontainer>/  /sourcekey:<key> /destkey:<key> /S

The above command will copy all blobs from the container named “sourcecontainer” in storage account “sourceaccount” to another container named “destcontainer” in storage account “destaccount”

If you have base blob with snapshots, please add/Snapshot to move all snapshots with base blob to destination, please be noted the blob snapshot will be renamed to this format in destination: [blob-name] (snapshot-time)[extension]

AzCopy https://<sourceaccount>.blob.core.windows.net/<sourcecontainer>/ https://<destaccount>.blob.core.windows.net/<destcontainer>/  /sourcekey:<key> /destkey:<key> /S /SNAPSHOT

For example if you have readme.txt with 3 snapshots in source container, then it will be as below in destination container
readme.txt

readme (2013-02-25 080757).txt
readme (2012-12-23 120657).txt
readme (2012-09-12 090521).txt

If you’d like to delete those blobs from the source container when the copy is complete, then just added /MOV as below

AzCopy https://<sourceaccount>.blob.core.windows.net/<sourcecontainer>/ https://<destaccount>.blob.core.windows.net/<destcontainer>/  /sourcekey:<key> /destkey:<key> /MOV /S

You can also create a response file to make it easier of running same command again and again. Create a txt file called “myAzCopy.txt” with content below

#URI of Source Container
https://<sourceaccount>.blob.core.windows.net/<sourcecontainer>/
#URI of Destination Container
https://<destaccount>.blob.core.windows.net/<destcontainer>/

Then you can run the command below to transfer files from source container to destination container

AzCopy </@:C:\myAzCopy.txt> /sourcekey:<key> /destkey:<key> /MOV /S