I have software (.ZIP) file in the blob storage as below in the azure, I want to download and install it in my local machine and also in another test VM. Please suggest how we can achieve this, thank you

Hi, UPDATE: Based on conversations in comments, Managed Identity is good option for your use case. One method, click Generate SAS tab in your screenshot, set the expire date/time to meet your needs, then Generate the SAS and URL. Now, the URL will allow you to download the file from both locations, assuming there are no networking restrictions on the storage account. If you are okay with manually installing then you can just open a browser in each, download, then extract files and install as required. If you need to be automated then you would need to write/test a PowerShell or other script to essentially do the above steps. If you need some pointers for creating the script let me know. Thanks. -TP

Hello Varma , Welcome to Microsoft Q&A Forum, Thank you for posting your query here! Adding more information to above answer: You could use AzCopy , a command-line utility, to copy files from Azure Blob storage to your local machine or an Azure VM. The easiest way to consider moving data from blob to VM SSD, you should use Azure Storage Explorer, this link provides the full documentation on how to use the tool. If you'd like an easy command line/code free solution allowing data transfer, i'd recommend ADF's copy wizard. You could specify the cloud source as the blob's location, and a destination as your VM. You can find a step-by-step tutorial in this link: https://www.red-gate.com/simple-talk/cloud/cloud-data/using-copy-wizard-azure-data-factory/ More can be found in this blog: https://azure.microsoft.com/en-us/blog/simple-and-reliable-data-movement-with-azure-data-factory-copy-wizard/ A suggestion i'd like to provide since I don't know the details and the architecture of your current setup, If you'd like something easier to manage in transferring data to VMs, I'd recommend using Azure file storage. It would be an easy solution for customers to add data by attaching the file share on their computer, and data would be directly stored in it. It can also be easily manipulated using Storage Explorer: https://learn.microsoft.com/en-us/azure/storage/files/storage-how-to-use-files-windows . Hope this helps! Please let us know if you have any further queries. I’m happy to assist you further. Please and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Using zip from storage in to the local machine/azureVM

Accepted answer

TP 90,306 Reputation points

2024-02-29T15:11:17.9733333+00:00

Hi,

UPDATE: Based on conversations in comments, Managed Identity is good option for your use case.

One method, click Generate SAS tab in your screenshot, set the expire date/time to meet your needs, then Generate the SAS and URL.

Now, the URL will allow you to download the file from both locations, assuming there are no networking restrictions on the storage account. If you are okay with manually installing then you can just open a browser in each, download, then extract files and install as required.

If you need to be automated then you would need to write/test a PowerShell or other script to essentially do the above steps. If you need some pointers for creating the script let me know.

Thanks.

-TP
Please sign in to rate this answer.
Varma 1,380 Reputation points

2024-02-29T15:18:22.9533333+00:00

HI TP, SAS is not the best option as the key is in plain text. The script needs to always work without editing. A clientID that can grab the secret form a keyvault is what I'm aiming for I have client id with me. Please suggest further. thank you.

TP 90,306 Reputation points

2024-02-29T15:23:41.3633333+00:00

Okay, that makes it more clear.
How about using managed identity, that way your script will not need to retrieve any kind of secret from key vault?

Varma 1,380 Reputation points

2024-02-29T15:26:35.0533333+00:00

Do we need to create managed identity? if yes what is the process to create in my scenario? at what level and what permission need to give? it requires loging in to azure account. That would need to be automated too if possible. I don't know if no touch zip file retreival is possible so far.

TP 90,306 Reputation points

2024-02-29T15:50:35.49+00:00

On the storage account blob container, you can assign Storage Blob Data Reader to the Managed Identity for the VM (you need to enable managed identity on the VM first).

For your local machine, is it okay to use your Entra ID to authenticate? You could write the script to use DefaultAzureCredential so that when it runs it automatically attempts to use different types of credentials in sequence. When you run it locally, the idea is it could get token under your credentials, so would use that. As above, your Entra ID would need to have access to the Blob.

When it runs in Azure VM, those creds wouldn't be available, but when it tries to get token for Managed Identity it would succeed and use that instead.

The idea is to have script stay same, but the type of credentials used to authenticate to the storage account would differ automatically.

Make sense? Or are you thinking of better/different approach?

Varma 1,380 Reputation points

2024-02-29T17:00:07.4133333+00:00

HI TP, Yes, first I would like to check with my entra id first, can you guide me where to start for that and how we can achive it?

Varma 1,380 Reputation points

2024-03-01T05:06:34.9266667+00:00

HI TP,

Just checking whether you got a chance to look above comment

TP 90,306 Reputation points

2024-03-01T05:14:17.42+00:00

will reply back within an hour or so and help you get it working

TP 90,306 Reputation points

2024-03-01T06:02:03.29+00:00

Once you confirm it works I will edit my answer above to have script + notes

Some notes:

On your local PC, you need to add environment variable bloblocal=yes After adding the environment variable close/re-open PowerShell window

You need to enable System Managed Identity for the VM via Identity blade

You need to assign Storage Blob Data Reader to the container for the system managed identity of the VM as well as the Entra ID that will be used to run it locally via Access Control (IAM) blade of the container

You need to make sure windows user account that runs script has access to the path where the file will be downloaded to as well as destination path for extracted files

Script is set to overwrite the downloaded file as well as destination extracted files

$storageAccount = "mystorageaccount" $containerName = "mycontainer" $blobName = "myfile.zip" $localFileName = "myfile.zip" $localDestination = "C:\Temp\myfileExtracted" if ($env:bloblocal -eq "yes") { Connect-AzAccount } else { Connect-AzAccount -Identity } $ctx = New-AzStorageContext -StorageAccountName $storageAccount -UseConnectedAccount Get-AzStorageBlobContent -Context $ctx -Container $containerName -Blob $blobName -Destination $localFileName -Force Expand-Archive -Path $localFileName -DestinationPath $localDestination -Force

Please let me know if something isn't clear

Varma 1,380 Reputation points

2024-03-01T06:38:20.79+00:00

HI TP,

I have not implemented your steps yet,

I tried to download blob using below azure cli command , please see below

PS C:\Windows\system32> az storage blob download --account-name ue2prdevinfstg --container-name pp-devinf-storage-blob --fileXXXX.zip --name Binaries/XXX5/XXXX.zip D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\cryptography/hazmat/backends/openssl/backend.py:27: UserWarning: You are using cryptography on a 32-bit Python on a 64-bit Windows Operating System. Cryptography will be significantly faster if you switch to using a 64-bit Python.

There are no credentials provided in your command and environment, we will query for account key for your storage account. It is recommended to provide --connection-string, --account-key or --sas-token in your command as credentials.

You also can add --auth-mode login in your command to use Azure Active Directory (Azure AD) for authorization if your login account is assigned required RBAC roles. For more information about RBAC roles in storage, visit https://docs.microsoft.com/azure/storage/common/storage-auth-aad-rbac-cli.

In addition, setting the corresponding environment variables can avoid inputting credentials in your command. Please use --help to get more information about environment variable usage. Finished[#############################################################] 100.0000% { "container": "pp-devinf-storage-blob", "content": "", "contentMd5": null, "deleted": false, "encryptedMetadata": null, "encryptionKeySha256": null, "encryptionScope": null, "hasLegalHold": null, "hasVersionsOnly": null,

now my question where is downloaded and how to install downloaded file? could you please suggest

TP 90,306 Reputation points

2024-03-01T06:48:54.2+00:00

the zip file you downloaded should be in C:\windows\system32 folder. You can use Expand-Archive command to extract the files to a new folder, something like below:

Expand-Archive myfile.zip -DestinationPath "C:\Temp\myfileExtracted"

Varma 1,380 Reputation points

2024-03-01T10:29:53.1566667+00:00

HI TP,

Please see below , it is not recognizing.

Please suggest

TP 90,306 Reputation points

2024-03-01T13:16:30.0633333+00:00

It looks like you are missing hyphen in front of file option. It needs to have two hyphens, like this:

--file "C:\Temp\DELPHI5\DELPHI5AUTO.zip"

Varma 1,380 Reputation points

2024-03-02T05:50:34.8433333+00:00

Hi TP,

I added --file, it seems it worked. but I see different error below, PLease suggest

Command executed: and error

az storage blob download --account-name ue2prdevinfstg --container-name pp-devinf-storage-blob --blob-url

https://XXXXXXXX/ppdevinf-storage-blob/Binaries/Delphi5/DELPHI5AUTO.zip

--file "C:\Temp\DELPHI5\DELPHI5AUTO.zip"

PS C:\Users\azureuser> C:\Users\azureuser\Documents\Untitled1.ps1

az : WARNING:

At C:\Users\azureuser\Documents\Untitled1.ps1:1 char:1

az storage blob download --account-name ue2prdevinfstg --container-na ...

+ CategoryInfo : NotSpecified: (WARNING: :String) [], RemoteException + FullyQualifiedErrorId : NativeCommandError

There are no credentials provided in your command and environment, we will query for account key for your storage account.

It is recommended to provide --connection-string, --account-key or --sas-token in your command as credentials.

You also can add --auth-mode login in your command to use Azure Active Directory (Azure AD) for authorization if your login account is assigned required RBAC roles.

For more information about RBAC roles in storage, visit

https://docs.microsoft.com/azure/storage/common/storage-auth-aad-rbac-cli.

In addition, setting the corresponding environment variables can avoid inputting credentials in your command. Please use --help to get more information about environment variable usage.

WARNING:

Skip querying account key due to failure: Please run 'az login' to setup account.

ERROR: Server failed to authenticate the request. Please refer to the information in the www-authenticate header.

RequestId:6fb44c14-f01e-0077-7464-6ca006000000

Time:2024-03-02T05:46:11.7358859Z

ErrorCode:NoAuthenticationInformation

I am doing above inside test virtual machine, now what is the way for me should be secured and automatic and dynamic to use credentials. please suggest and looking forword response from you, thank you

TP 90,306 Reputation points

2024-03-02T06:15:08.08+00:00

Hi,

First, you need to enable System Managed Identity on the VM. Navigate to the VM in the portal, Identity blade. Click ON, then click Save button, Yes.

Second, navigate to the storage account, click Containers blade, next click on the container where the file is on right. On the Access control (IAM) blade, click Add role assignment, select Storage Blob Data Reader, Next, select Managed Identity, click Select members, select the VM's managed identity, click Select, click Review + create, Create.

Third, run the script I posted above in the VM (set parameters at top before running):

$storageAccount = "mystorageaccount" $containerName = "mycontainer" $blobName = "myfile.zip" $localFileName = "myfile.zip" $localDestination = "C:\Temp\myfileExtracted" if ($env:bloblocal -eq "yes") { Connect-AzAccount } else { Connect-AzAccount -Identity } $ctx = New-AzStorageContext -StorageAccountName $storageAccount -UseConnectedAccount Get-AzStorageBlobContent -Context $ctx -Container $containerName -Blob $blobName -Destination $localFileName -Force Expand-Archive -Path $localFileName -DestinationPath $localDestination -Force

When you run it you will see you won't need to enter credentials since it will automatically use System Managed Identity.

Varma 1,380 Reputation points

2024-03-02T06:28:18.69+00:00

HI TP,

Looks like I don't have access to add role assignments. am I correct?

1.

now what is the exact reason I should give to management to get the access?

or is there any other way to achive this with out this process?

2.

and is there any way that I can download this package to github and use it from there? please let me know your prospective on this. because in the end I want to use it in the packer provisoner and to execut delphi.ps1 file.

TP 90,306 Reputation points

2024-03-02T17:53:49.6933333+00:00

Correct, you don't have permission to add role assignments on the storage account. I was asking you to add role assignment on the specific Container, not the storage account as a whole. I'm guessing you don't have permission to add role assignment to a container either.

now what is the exact reason I should give to management to get the access? or is there any other way to achive this with out this process?

If you want to download the .zip from blob storage then the identity that will be used needs to have permission to download it. You could use a User Managed Identity (UMI) instead, however, your management will need to add role assignment to the blob Container for the User Managed Identity. Additionally you (or whatever account) will need Managed Identity Operator role on the UMI so that you can assign it to the VM.> and is there any way that I can download this package to github and use it from there? please let me know your prospective on this. because in the end I want to use it in the packer provisoner and to execut delphi.ps1 file.

One alternative would be to put the .zip file on public github repro as you mentioned so you could download it from there. If it is okay that the .zip file be available publicly like that then you could use this option instead.

Another similar alternative would be to make the .zip file available anonymously in the storage account container. In other words, the storage account container would be configured to allow the .zip file to be downloaded without any authentication.

Varma 1,380 Reputation points

2024-03-03T05:31:52.86+00:00

HI TP.

It is private repo and private zip file, so seems I can not use both git hub and storage account anonymous option

I would like to create user managed identity to the virtual machine which gets created in the backend and gets deleted in the backend. COuld you please tell me how to create user managed identity before itself?

https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/how-to-use-vm-token#get-a-token-using-powershell

Varma 1,380 Reputation points

2024-03-03T05:36:16.7633333+00:00

HI TP,

I raised a new ticket as well, please assist on this

https://learn.microsoft.com/en-us/answers/questions/1606850/creating-user-managed-identity-considering-secure

TP 90,306 Reputation points

2024-03-03T06:23:34.4866667+00:00

It is private repo and private zip file, so seems I can not use both git hub and storage account anonymous option I would like to create user managed identity to the virtual machine which gets created in the backend and gets deleted in the backend. COuld you please tell me how to create user managed identity before itself?

The User Managed Identity (UMI) needs to be created, then you need to ask them to assign the Storage Blob Data Reader role to the container in the storage account. On the UMI you (or whichever account that will be used to create VM) need to be assigned Managed Identity Operator role.

Creating the UMI, assigning roles, is a one time thing. Once that is done, we just need to make a small change to the script I gave you above and it will use the UMI instead of System Managed Identity.

When you create the VM it needs to be assigned the UMI.

Does that make sense?

Varma 1,380 Reputation points

2024-03-03T07:11:24.1066667+00:00

Hi TP,

I have created user managed identity as below but it is available at the Resoruce group level it seems

can we create a this identity at the storage account blob level ?

and where is the option to assign?

TP 90,306 Reputation points

2024-03-03T10:03:02.65+00:00

You need to Add Role Assignment to the Container, however, above you showed that you don't have permission to do this. This means you need to have someone else add the role assignment to the Container.

In other words, someone else would navigate to the storage account -- Containers -- <yourcontainer> -- Access control (IAM) blade and add the role assignment for Storage Blob Data Reader and select the User Managed Identity that you created

Varma 1,380 Reputation points

2024-03-03T10:56:58.5166667+00:00

HI TP,

I got it,

Could you please clarify my following questios

After added respective role assignment to storage account that error is gone

but now it is giving add az login error in the packer logs.

can we add az login with out asking for prompt in the below template. I know below will give prompt while executing in the packer build

2nd question

since we create managed identity at the resource group level and every body can use that managed identity for accessing some other resources and by adding role, what is the way to fix this? please suggest.

3rd question

do you think using access token, is it needed in my case? if yes please suggest how to fix following error

Varma 1,380 Reputation points

2024-03-05T06:21:45.1466667+00:00

HI TP,

It worked, you can make your main answer as accepted answer
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.