Copy / Move data from Azure Storage account file share to blob container

Ghulam Abbas 151 Reputation points
2024-03-06T17:53:32.21+00:00

We have some data in Azure storage account as file share that we would like to copy / move to blob container within the same storage account. We have looked into a few ways to do this and found AZ Copy would be the best option to achieve this (as per MS official documentation).

When we try to use az copy commands to copy data from file share to blob container, we are getting an error (although we added the clinet IP to the firewall and allowed the subnets under Networking section): Any suggestion / advice would be appreciated:

INFO: Scanning... INFO: Failed to create one or more destination container(s). Your transfers may still succeed if the container already exists. INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support INFO: Failed to scan directory/file . Logging errors in scanning logs.

failed to perform copy command due to error: cannot start job due to error: -> github.com/Azure/azure-storage-file-go/azfile.newStorageError, /usr/src/mariner/BUILD/azure-storage-azcopy-10.15.0/vendor/github.com/Azure/azure-storage-file-go/azfile/zc_storage_error.go:42 ===== RESPONSE ERROR (ServiceCode=AuthorizationFailure) ===== Description=This request is not authorized to perform this operation. RequestId:ec4d4d49-001a-000a-7cc5-6f710f000000 Time:2024-03-06T12:58:09.5181299Z, Details: Code: AuthorizationFailure

User-Agent: [AzCopy/10.15.0 Azure-Storage/0.8.0 (go1.20.10; linux)] X-Ms-Client-Request-Id: [6b7ef3d1-32c1-4abe-6ce9-5f83b56b85ea] X-Ms-Version: [2020-10-02] -------------------------------------------------------------------------------- RESPONSE Status: 403 This request is not authorized to perform this operation. Content-Length: [246] Content-Type: [application/xml] Date: [Wed, 06 Mar 2024 12:58:09 GMT] Server: [Microsoft-HTTPAPI/2.0] X-Ms-Client-Request-Id: [6b7ef3d1-32c1-4abe-6ce9-5f83b56b85ea] X-Ms-Error-Code: [AuthorizationFailure] X-Ms-Request-Id: [ec4d4d49-001a-000a-7cc5-6f710f000000]

Azure Files
Azure Files
An Azure service that offers file shares in the cloud.
1,152 questions
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,647 questions
Azure Virtual Network
Azure Virtual Network
An Azure networking service that is used to provision private networks and optionally to connect to on-premises datacenters.
2,118 questions
0 comments No comments
{count} vote

Accepted answer
  1. Amrinder Singh 1,655 Reputation points Microsoft Employee
    2024-03-06T19:12:33.7+00:00

    Hi @Ghulam Abbas - Thanks for reaching out.

    If we are using right set of auth i.e. SAS for authentication on File share and SAS/Oauth on the blob storage, with the desired permissions, it could be further failing on the Networking plane.

    https://learn.microsoft.com/en-us/azure/storage/common/storage-ref-azcopy-copy

    Now, you mentioned that you have whitelisted the IP however it is quite possible that the actual IP hitting the storage is different than the whitelisted one and hence the failure. A quick test can be done by allowing access from the network and test the same. If it works then we are on right track.

    You can then try enabling the diagnostic logging to compare the client IP's in the failure and the successful scenario.

    Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.


2 additional answers

Sort by: Most helpful
  1. Marcelo Zambrana 0 Reputation points Microsoft Employee
    2024-03-06T18:42:43.96+00:00

    Could you post the complete command you are using?
    Also make sure to do az login in the client VM you are using, and make sure the user has privileges to do that storage account. By error it seems it does not.

    0 comments No comments

  2. Ghulam Abbas 151 Reputation points
    2024-03-07T14:25:26.2533333+00:00

    Just to mention, we are running these AZ Copy commands from Azure cloudshell (not if this makes any difference)

    0 comments No comments