it's must using SAS token for copy file between blob and blob?

Huang, Winston-HR 280 Reputation points
2024-02-27T08:52:40.4233333+00:00

we want to use azcopy tool to copy file between storage account's blob, and use service principal as login identity, when we try to run azcopy copy command without SAS token of storage account ,the job is copy files failed. it's must using SAS token for copy file between blob and blob?

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,348 questions
0 comments No comments
{count} votes

Accepted answer
  1. Ramya Harinarthini_MSFT 5,306 Reputation points Microsoft Employee
    2024-02-27T09:43:46.6933333+00:00

    @Huang, Winston-HR - Welcome to Microsoft Q&A Forum, thank you for posting your query here!! Make sure you service principle have proper RBAC role i.e Storage blob data contributor in order to perform the copy operation across storage accounts. Reference Link: https://learn.microsoft.com/en-us/azure/storage/common/storage-ref-azcopy-login#examples You can follow below steps to perform the copy operation through Azcopy using service principal. 1. Creating a service principal To create a service principal we will use Cloud Shell on Azure Portal using the az ad sp create-for-rbac command. The below command will provide an Azure Storage data access role to assign to the new service principal. Additionally, provide the scope for the role assignment. For more information about the built-in roles provided for Azure Storage, see Azure built-in roles. Note: Save the output of the create SPN command.az ad sp create-for-rbac `
     --name <service-principal> `
     --role "Storage Blob Data Contributor" `
     --scopes /subscriptions/<subscription>/resourceGroups/<resource-group>/providers/Microsoft.Storage/storageAccounts/<storage-account>thumbnail image 1 captioned Creating service principalCreating service principal Assigning roles to service principal Once the Service Principal is created, we also need to grant ‘Reader’ role on the storage account to the service principal. This will grant the SPN read access to storage resource at subscription level. Please refer to our documentation on assigning roles for access to blob. https://docs.microsoft.com/en-us/azure/storage/common/storage-auth-aad-rbac-portal az role assignment create --assignee "<appId>" `
     --role "Reader" `
     --scope "/subscriptions/<subscription>/resourceGroups/<resource-group>/providers/Microsoft.Storage/storageAccounts/<storage-account>"thumbnail image 2 captioned Role assignmentsRole assignments 2. Using service principal with AzCopy AzCopy is a command-line tool that moves data into and out of Azure Storage. To learn more about AzCopy please refer the official documentation. Login as service principal Next we will login as the service principal in AzCopy using the azcopy login command. The values for options application-id, tenant-id and AZ_COPY_CLIENT_SECRET, will be available on step 1 after creating the service principal.$env:AZCOPY_SPA_CLIENT_SECRET="$(Read-Host -prompt "Enter key")"

    azcopy login `
     --service-principal `
     --application-id "<appId>" `
     --tenant-id "<tenantId>"thumbnail image 3 captioned AzCopy loginAzCopy login Performing copy operations Once sucessfully logged in, we can upload and download files using OAuth authentication of the service principal with azcopy copy command.

    Upload exampleazcopy copy "/path/to/file.txt" "https://[account].blob.core.windows.net/[container]/[path/to/blob]"thumbnail image 4 captioned Upload blob with AzCopyUpload blob with AzCopy Hope this helps!
    Kindly let us know if the above helps or you need further assistance on this issue. -------------------------------------------------------------------------------------------------------------please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.  

    1 person found this answer helpful.
    0 comments No comments

1 additional answer

Sort by: Most helpful
  1. Amrinder Singh 2,195 Reputation points Microsoft Employee
    2024-02-27T09:13:51.89+00:00

    Hi @Huang, Winston-HR - Welcome to Microsoft Q&A Forum, thank you for posting your query here!

    Below are the auth method possible when using AzCopy Tool based on the account type:

    • Azure Blob (SAS or public) -> Azure Blob (SAS or OAuth authentication)
    • Azure Data Lake Storage Gen2 (SAS or public) -> Azure Data Lake Storage Gen2 (SAS or OAuth authentication)
    • Azure Blob (SAS or OAuth authentication) <-> Azure Blob (SAS or OAuth authentication) - See Guidelines.
    • Azure Data Lake Storage Gen2 (SAS or OAuth authentication) <-> Azure Data Lake Storage Gen2 (SAS or OAuth authentication)
    • Azure Data Lake Storage Gen2 (SAS or OAuth authentication) <-> Azure Blob (SAS or OAuth authentication)

    https://learn.microsoft.com/en-us/azure/storage/common/storage-ref-azcopy-copy

    There is OAuth option available too with some of the guidelines to review such as both accounts part of same tenant and also having access on the networking plane ahead.

    Kindly let us know if you have any further queries. I’m happy to assist you further. Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

    1 person found this answer helpful.