Copy / Move data from Azure Storage account file share to blob container

Question

We have some data in Azure storage account as file share that we would like to copy / move to blob container within the same storage account. We have looked into a few ways to do this and found AZ Copy would be the best option to achieve this (as per MS official documentation).

When we try to use az copy commands to copy data from file share to blob container, we are getting an error (although we added the clinet IP to the firewall and allowed the subnets under Networking section): Any suggestion / advice would be appreciated:

INFO: Scanning... INFO: Failed to create one or more destination container(s). Your transfers may still succeed if the container already exists. INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support INFO: Failed to scan directory/file . Logging errors in scanning logs.

failed to perform copy command due to error: cannot start job due to error: -> github.com/Azure/azure-storage-file-go/azfile.newStorageError, /usr/src/mariner/BUILD/azure-storage-azcopy-10.15.0/vendor/github.com/Azure/azure-storage-file-go/azfile/zc_storage_error.go:42 ===== RESPONSE ERROR (ServiceCode=AuthorizationFailure) ===== Description=This request is not authorized to perform this operation. RequestId:ec4d4d49-001a-000a-7cc5-6f710f000000 Time:2024-03-06T12:58:09.5181299Z, Details: Code: AuthorizationFailure

User-Agent: [AzCopy/10.15.0 Azure-Storage/0.8.0 (go1.20.10; linux)] X-Ms-Client-Request-Id: [6b7ef3d1-32c1-4abe-6ce9-5f83b56b85ea] X-Ms-Version: [2020-10-02] -------------------------------------------------------------------------------- RESPONSE Status: 403 This request is not authorized to perform this operation. Content-Length: [246] Content-Type: [application/xml] Date: [Wed, 06 Mar 2024 12:58:09 GMT] Server: [Microsoft-HTTPAPI/2.0] X-Ms-Client-Request-Id: [6b7ef3d1-32c1-4abe-6ce9-5f83b56b85ea] X-Ms-Error-Code: [AuthorizationFailure] X-Ms-Request-Id: [ec4d4d49-001a-000a-7cc5-6f710f000000]

Answer 1

Hi @Ghulam Abbas - Thanks for reaching out.

If we are using right set of auth i.e. SAS for authentication on File share and SAS/Oauth on the blob storage, with the desired permissions, it could be further failing on the Networking plane.

https://learn.microsoft.com/en-us/azure/storage/common/storage-ref-azcopy-copy

Now, you mentioned that you have whitelisted the IP however it is quite possible that the actual IP hitting the storage is different than the whitelisted one and hence the failure. A quick test can be done by allowing access from the network and test the same. If it works then we are on right track.

You can then try enabling the diagnostic logging to compare the client IP's in the failure and the successful scenario.

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Answer 2

Could you post the complete command you are using?
Also make sure to do az login in the client VM you are using, and make sure the user has privileges to do that storage account. By error it seems it does not.

Answer 3

Just to mention, we are running these AZ Copy commands from Azure cloudshell (not if this makes any difference)