Exercise - Move Azure Storage blob using AzCopy - Training

6 minutes

You decided to evaluate the AzCopy command as an alternative to using the Azure CLI. As before, you want to move blobs that contain the details for product specifications, and that are more than six months old, to an archive store in a separate Blob Storage account.

In this exercise, you investigate how to use AzCopy to migrate blobs that weren't changed in the last six months to a separate storage account.

Create and add data to hot storage

First, create two accounts by using the Azure CLI.

The free sandbox allows you to create resources in a subset of the Azure global regions. Select a region from this list when you create resources:

westus2
southcentralus
centralus
eastus
westeurope

southeastasia
japaneast
brazilsouth
australiasoutheast
centralindia

Create environment variables for your storage account name and region. Replace <location> with a region from the previous list.
```
HOT_STORAGE_NAME=hotstorage$RANDOM
COOL_STORAGE_NAME=coolstorage$RANDOM
LOCATION=<location>
```

Next, run the following command to create a storage account to hold blobs.

az storage account create \
  --location $LOCATION \
  --name $HOT_STORAGE_NAME \
  --resource-group "<rgn>[Sandbox resource group name]</rgn>" \
  --sku Standard_RAGRS \
  --kind BlobStorage \
  --access-tier Hot

Create a storage account for holding the archived blobs. Use the Cool access tier. As before, specify an appropriate region, and choose a unique name for your storage account.

az storage account create \
  --location $LOCATION \
  --name $COOL_STORAGE_NAME \
  --resource-group "<rgn>[Sandbox resource group name]</rgn>" \
  --sku Standard_RAGRS \
  --kind BlobStorage \
  --access-tier Cool

Setup

Let's start by downloading the latest version of AzCopy; currently version 10. The Azure CLI does include AzCopy, but it might be an older version. Let's also generate SAS tokens for our storage accounts and then upload some sample data.

In the Cloud Shell window, run the following commands to download and extract the most recent version of AzCopy for Linux.
```
wget -O azcopy.gz https://aka.ms/downloadazcopy-v10-linux
gunzip azcopy.gz
tar xvf azcopy
```

Select this link to sign in to the Azure portal. This link signs you in to the Azure portal by using the same account with which you activated the sandbox.
Select All resources, and select your destination (cool) storage account.
On the storage account page, under Security + networking, select Shared access signature.
On the shared access signature page, check Container and Object under the Allowed resource types section.
With the allowed resource types checked, select Generate SAS and connection string at the bottom of the shared access signature page.
In the Cloud Shell, create an environment variable and assign the generated SAS token value from the portal.
```
COOL_SAS_TOKEN="<token from portal>"
```
Repeat these steps, and assign a generated SAS token for the hot storage account to a variable named HOT_SAS_TOKEN.
```
HOT_SAS_TOKEN="<token from portal>"
```

Run the following command to download the sample files for this exercise.

git clone https://github.com/MicrosoftDocs/mslearn-copy-move-blobs-from-containers-or-storage-accounts sample

Create a container for holding archived blobs

In the Cloud Shell, navigate to the directory containing the AzCopy utility, replacing <version> with the version number downloaded with the latest AzCopy release. You can find the directory name with the dir command.
```
cd azcopy_linux_amd64_<version>
```
Create a new container named azcopy-archive in the cool storage destination for the archived content from this AzCopy command line process.
```
./azcopy make https://$COOL_STORAGE_NAME.blob.core.windows.net/azcopy-archive$COOL_SAS_TOKEN
```
Verify that the command responds with the message, Successfully created the resource.
Create a new container named specifications in the hot storage destination for holding fictitious specification files.
```
./azcopy make https://$HOT_STORAGE_NAME.blob.core.windows.net/specifications$HOT_SAS_TOKEN
```
Verify that the command responds with the message, Successfully created the resource.

Upload the files to your storage account and save each one as a blob. This command uploads several specification files.

./azcopy copy "../sample/specifications/*" "https://$HOT_STORAGE_NAME.blob.core.windows.net/specifications$HOT_SAS_TOKEN"

Copy blobs to Cool storage

Run the following command in the Cloud Shell, to generate a list of blobs and their last modified times in the source container.

az storage blob list \
  --account-name $HOT_STORAGE_NAME \
  --container-name "specifications" \
  --sas-token $HOT_SAS_TOKEN \
  --query '[].{name:name, properties:properties.lastModified}'

Now that you have a list of blobs and their last modified times, you can use this information to create a script. The script uses AzCopy to transfer each blob that has a last modified time of more than six months ago to Cool storage. Because you just added these files, none of them are old enough to filter by six months. Instead, run this example command to copy the specification01.md file to this new cool storage container.
```
./azcopy copy https://$HOT_STORAGE_NAME.blob.core.windows.net/specifications/specification01.md$HOT_SAS_TOKEN https://$COOL_STORAGE_NAME.blob.core.windows.net/azcopy-archive$COOL_SAS_TOKEN
```
Copying a blob that is small, causes the transfer to complete quickly, but imagine that the blob is several gigabytes in size. Run the following command to view AzCopy jobs that ran recently, or are still running.
```
./azcopy jobs list
```
Make a note of the JobId for the transfer operation.
Run the following command to determine the status of the job. Replace <jobid> with the job ID that you noted in the previous step.
```
./azcopy jobs show <jobid>
```
Verify that the blob was copied. Run the following command to list the blobs in the azcopy-archive container in the destination storage account.
```
./azcopy list https://$COOL_STORAGE_NAME.blob.core.windows.net/azcopy-archive$COOL_SAS_TOKEN
```

Remove blobs from Hot storage

Copying a blob to Cool storage doesn't remove the blob from Hot storage. You must delete these blobs manually. As an example, run the following command to delete the "specification01.md" blob from the source (hot) storage account.
```
./azcopy remove https://$HOT_STORAGE_NAME.blob.core.windows.net/specifications/specification01.md$HOT_SAS_TOKEN
```

Continue

Exercise - Move Azure Storage blobs using AzCopy

Create and add data to hot storage

Setup

Create a container for holding archived blobs

Copy blobs to Cool storage

Remove blobs from Hot storage

Feedback