send blob file to gcp storage

Question

I have to send file from azure blob to gcp storage bucket . I want to use azure powershell based function and using workload identity federation as authentication. Could you please provide some reference of powershell commands

Answer 1

Hi @techazure Welcome to the Microsoft Q&A and thank you for posting your questions here. Sequel to your question, you would like to send a file from Azure blob to GCP storage bucket and also would want to use azure PowerShell based function and using workload identity federation as authentication. To transfer files from Azure Blob Storage to Google Cloud Storage using PowerShell and Workload Identity Federation for authentication you will need to follow a series of steps. Here's my suggested processes.

1. Install the PowerShell modules:

   Install-Module -Name Az.Storage -Force -AllowClobber
   

1. Set up Workload Identity Federation: Make sure you have Workload Identity Federation configured correctly on both Azure and GCP. This involves creating service accounts assigning roles and configuring permissions as, per your requirements.
2. Configure your credentials: Set up your Azure credentials using Workload Identity Federation and Configure Google Cloud Credentials. The specific commands may vary depending on your setup. Generally you will need to use the "az login" and "gcloud auth login" command. Your code will similar to:

   az login --identity
   

   gcloud auth login
   
   

1. Now, you can use PowerShell commands to copy files from Azure Blob Storage to Google Cloud Storage.

   # Download file from Azure Blob Storage
   $storageAccountName = "<your_storage_account_name>"
   $containerName = "<your_container_name>"
   $blobName = "<your_blob_name>"
   $localFilePath = "C:\Temp\downloaded_file.txt"
   az storage blob download --account-name $storageAccountName --container-name $containerName --name $blobName --file $localFilePath
   # Upload file to Google Cloud Storage
   $bucketName = "<your_gcp_bucket_name>"
   $gcpFilePath = "gs://$bucketName/uploaded_file.txt"
   gcloud auth application-default login
   gcloud storage cp $localFilePath $gcpFilePath
   # Make sure to replace the placeholders <your_storage_account_name>, <your_container_name>, <your_blob_name>, and <your_gcp_bucket_name> with your actual Azure and GCP storage details.
   

Please note that implementing Workload Identity Federation and executing the corresponding commands can be slightly complex depending on your setup. Kindly, modify the above code tailor to your use case. I hope this is helpful! Do not hesitate to let me know if you have any other questions. Please remember to "Accept Answer" if answer helped, so that others in the community facing similar issues can easily find the solution. Best Regards, Sina Salam

Answer 2

@Neha Bameta There are multiple aspects to consider here and I will try to list everything you would need based on what I know but do note that is a scenario that I haven't worked with before

1. First you would need to set up the Workload Identity Federation, if you haven't already. The docs go into enough detail to set things up. If you have any specific questions, they warrant a new post specific to that.
2. To get the Token in PowerShell, there is a discussion about it in this issue, which does not seem to have perfect support yet but can still be leveraged to work in a certain way. The -FederatedToken parameter is what you need, to which you pass in the token from GCP, and other details to in turn fetch the token for Azure Entra ID.
3. With the identity part handled, you can now directly use the Azure Storage PowerShell cmdlets like Get-AzStorageBlob to fetch your blobs
4. You already have the token from GCP, so I suppose you could simply use the New-GcsObject cmdlet to then upload the downloaded file.