Moving data from Azure Virtual machine to Storage account within same subscription

Mazurek, Jakub 60 Reputation points
2024-11-25T14:52:04.44+00:00

Hi!

Currently we are using the build system on the Virtual machine running on linux in azure and send the data ( full folders into ) storage account in the same subscription via mounted storage with smb module, but sometimes we run into issues with slot swapping, checking, moving files etc. so we wanted to move to azure native solution.

Now, I've checked how to do it and it seems like it needs to use SAS token and I would like to avoid that, can it be done with some other way that wouldn't require timed tokens etc? It's in the same environment after all.

Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
3,268 questions
Azure ExpressRoute
Azure ExpressRoute
An Azure service that provides private connections between Azure datacenters and infrastructure, either on premises or in a colocation environment.
391 questions
Microsoft Entra ID
Microsoft Entra ID
A Microsoft Entra identity service that provides identity management and access control capabilities. Replaces Azure Active Directory.
22,462 questions
{count} votes

Accepted answer
  1. Vinodh247 25,291 Reputation points MVP
    2024-11-25T17:01:34.6+00:00

    Hi ,

    Thanks for reaching out to Microsoft Q&A.

    To move data from an Azure VM to a Storage Account without using SAS tokens, you can leverage Azure Managed Identity or Azure AD authentication. These methods eliminate the need for SAS tokens and provide secure, identity-based access to your storage resources. Here's how you can do it:

    Option 1: Use Managed Identity

    1. Enable Managed Identity for the VM:
      • In the Azure Portal, go to your VM's settings.
        • Under the "Identity" section, enable the System-Assigned Managed Identity.
        1. Assign Role to the VM:
          • In the Storage Account, go to Access Control (IAM).
            • Assign the "Storage Blob Data Contributor" role to the VM's managed identity. This role provides permissions to read/write to blob storage.
            1. Mount the Storage Account Using Azure CLI or SDK:
              • Install the Azure CLI on your VM (if not already installed).
                • Use azcopy or a script to copy files to the storage account, authenticating via the managed identity. Example:

    azcopy copy "/path/to/local/files" "https://<storage-account-name>.blob.core.windows.net/<container-name>" --recursive

    Option 2: Azure AD Authentication with Storage SDKs

    1. Configure Azure AD Authentication:
      • Follow the steps above to enable Managed Identity and assign roles.
        • Use the Azure SDK for Python, .NET, or any other supported language to authenticate using Azure AD. For example, in Python:

    from azure.identity import DefaultAzureCredential from azure.storage.blob import BlobServiceClient   credential = DefaultAzureCredential() blob_service_client = BlobServiceClient(account_url="https://<storage-account-name>.blob.core.windows.net", credential=credential)   container_client = blob_service_client.get_container_client("<container-name>") blob_client = container_client.get_blob_client("example.txt")   with open("example.txt", "rb") as data:     blob_client.upload_blob(data)

    Benefits of Managed Identity and Azure AD Authentication:

    • No Tokens: Avoid the complexity and security risks of managing SAS tokens.
    • Granular Permissions: Use Azure Role-Based Access Control (RBAC) to assign specific permissions.
    • Secure and Scalable: Authentication and authorization are handled securely by Azure.

    Common Tools for File Transfer:

    • Azure CLI: Use az storage blob upload commands with managed identity authentication.
    • AzCopy: Use for efficient and scalable file transfers.
    • Azure File Share with SMB: If you need a mounted drive, you can authenticate with Azure AD.

    Please feel free to click the 'Upvote' (Thumbs-up) button and 'Accept as Answer'. This helps the community by allowing others with similar queries to easily find the solution.

    1 person found this answer helpful.
    0 comments No comments

1 additional answer

Sort by: Most helpful
  1. Sreeju Nair 12,376 Reputation points
    2024-11-25T16:58:36.77+00:00

    Yes, you can move to an Azure-native solution for transferring data to your Azure Storage Account without relying on SAS tokens. Instead, you can leverage Managed Identities or Access Keys for authentication and access.

    Managed Identities provide a secure and seamless way for Azure services to authenticate and access other Azure resources without managing credentials or SAS tokens.

    Refer: https://learn.microsoft.com/en-us/azure/storage/blobs/authorize-access-azure-active-directory

    1 person found this answer helpful.
    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.