How do you programmatically copy an Azure storage container to a new container and delete the old container with/without using AZCOPY in code using VB or C#?

Question

I store files/images by address. Sometimes addresses can change for our clients/clients and when they do it breaks the link for the images stored in Azure. I'm trying to figure out how to copy an entire container/directories and blobs from the old address container and create a new container and copy everything from the old container, directories and blobs into the new address container when the address change happens programmatically. Any help is appreciated.

Answer 1

Hi @MR BILL

Thank you for asking this question on the **Microsoft Q&A Platform. **

The following link has the documentation for the REST API for the Storage Account https://learn.microsoft.com/en-us/rest/api/storagerp/storage-accounts

This is Azure Storage REST API Reference https://learn.microsoft.com/en-us/rest/api/storageservices/

Hope this helps,

----------

Accept Answer and Upvote, if any of the above helped, this thread can help others in the community looking for remediation for similar issues.
NOTE: To answer you as quickly as possible, please mention me in your reply.

Answer 2

@MR BILL Thank you for reaching out to Microsoft Q&A. I understand that you want to be able to copy contents of your Azure Container i.e., all the directories, Blobs etc., to a new container after creating the same when your address change happens, programmatically.

As you know the best way to do that would be via AzCopy as mentioned here- https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-blobs-copy#copy-a-container

However, I understand that you are looking for a programmatic way to do the same. Currently we have documentation for copying Blobs between storage accounts as seen here- https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-dotnet-legacy?source=recommendations#download-blobs

You are looking to copy the entire container. Since we do not have a standardized script for this purpose at the point and therefore has to be customized as needed. I will be attaching an example script soon that you can refer to which will be doing similar functions i.e., create/copy containers/blobs. Hope it helps.

Please let us know if you have any more questions and we will be glad to assist you further. Thank you!

Remember:

Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.

Want a reminder to come back and check responses? Here is how to subscribe to a notification.

Answer 3

@MR BILL Adding more information to the above response! of @Sai

There’s no(one) call way to copy the container, you would have to first list all the blobs in the container and then copy each and every blob into the new container,
“Can you copy a container or not? Everything I've found so far talks about copying blobs from the container, but not the container and all it's contents.”
Firstly we have to list all the blobs in the container and copy each and every one of them over.

However their scenario of the “address” / name of the blobs changing during the copy is worrisome to me because then the original list call of all the blob paths
would become invalid since they change the name of the blobs midflight of copying other blobs.
We shouldn't change the name of the blobs while a copy is occurring, or you have to keep listing all the blobs in the container in order to get a current version of the container.

Copy and move blobs from one container or storage account to another from the command line and in code: https://learn.microsoft.com/en-us/training/modules/copy-blobs-from-command-line-and-code/

Azcopy is the best option for your scenario! AzCopy v10 is just as fast as Fast Data Transfer, but is easier to set up. Customers should use AzCopy v10 in most cases. Fast Data Transfer runs on Windows and Linux. Its client-side portion is a command-line application that runs on-premises, on your own machine. A single client-side instance supports up to 10 Gbps. Its server-side portion runs on Azure VM(s) in your own subscription. Depending on the target speed, between 1 and 4 Azure VMs are required. An Azure Resource Manager template is supplied to automatically create the necessary VM(s). The tool works by maximizing utilization of the network link. It efficiently uses all available bandwidth, even over long-distance links. On a 10 Gbps link, it reaches around 4 TB per hour, which makes it about 3 to 10 times faster than competing tools we’ve tested. On slower links, Fast Data Transfer typically achieves over 90% of the link’s theoretical maximum, while other tools may achieve substantially less

This article provides an overview of some of the common Azure data transfer solutions. The article also links out to recommended options depending on the network bandwidth in your environment and the size of the data you intend to transfer. Choose an Azure solution for data transfer based on your requirements.

There are several options for transferring data to and from Azure, depending on your needs. Transfer data to and from Azure

We had to copy a bunch of containers from one storage account to another recently, and needed to be able to do it repeatedly, incrementally, and programmatically, we wrote some PowerShell functions to do it: (see CreateContainers, CopyContainers, and CopyBlobs functions).

This is the most optimal way to do it; however, some of the containers were quite large so we had to incorporate background copying of large batches of files to get decent performance, so in that sense, it’s probably pretty good for what it does. (By background copying, we mean to tell the storage system to do the copy directly from storage account to storage account, the data never travels to the system running the program – which avoids a huge overhead. This is different than the sample code you linked to which is focused on uploading and downloading, which would make the local system a bottleneck.)

You will probably notice it’s not directly copying folders. That’s because in the standard storage setup folders don’t really exist, they’re just a grouping of files with the same name (in other words, copying the files implicitly copies the ‘folders’ and there’s no way to have an empty folder). That may be different for hierarchical storage, and it’s definitely different for files. The code only works for blobs, and We have only tested it on non-hierarchical storage

Please let us know if you have any further queries. I’m happy to assist you further.

----------

Please do not forget to and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.