Edit

Example: Access Azure Storage using the Azure libraries for Python

  • Article

In this article, you learn how to use the Azure client libraries in Python application code to upload a file to an Azure Blob storage container. The article assumes you've created the resources shown in Example: Create Azure Storage.

All the commands in this article work the same in Linux/macOS bash and Windows command shells unless noted.

1: Set up your local development environment

If you haven't already, set up an environment where you can run this code. Here are some options:

2: Install library packages

In your requirements.txt file, add lines for the client library package you'll use and save the file.

azure-storage-blob
azure-identity

Then, in your terminal or command prompt, install the requirements.

pip install -r requirements.txt

3: Create a file to upload

Create a source file named sample-source.txt. This file name is what the code expects.

Hello there, Azure Storage. I'm a friendly file ready to be stored in a blob.

4: Use blob storage from app code

The following two sections demonstrate two ways to access the blob container created through Example: Create Azure Storage.

The first method (with authentication) authenticates the app with DefaultAzureCredential as described in Authenticate Python apps to Azure services during local development using service principals. With this method, you must first assign the appropriate permissions to the app identity, which is the recommended practice.

The second method (with connection string) uses a connection string to access the storage account directly. Although this method seems simpler, it has two significant drawbacks:

  • A connection string inherently authenticates the connecting agent with the Storage account rather than with individual resources within that account. As a result, a connection string grants broader authorization than might be needed.

  • A connection string contains access info in plain text and therefore presents potential vulnerabilities if it's not properly constructed or secured. If such a connection string is exposed, it can be used to access a wide range of resources within the Storage account.

For these reasons, we recommend using the authentication method in production code.

4a: Use blob storage with authentication

  1. Create a file named use_blob_auth.py with the following code. The comments explain the steps.

    import os
import uuid

from azure.identity import DefaultAzureCredential

# Import the client object from the SDK library
from azure.storage.blob import BlobClient

credential = DefaultAzureCredential()

# Retrieve the storage blob service URL, which is of the form
# https://<your-storage-account-name>.blob.core.windows.net/
storage_url = os.environ["AZURE_STORAGE_BLOB_URL"]

# Create the client object using the storage URL and the credential
blob_client = BlobClient(
    storage_url,
    container_name="blob-container-01",
    blob_name=f"sample-blob-{str(uuid.uuid4())[0:5]}.txt",
    credential=credential,
)

# Open a local file and upload its contents to Blob Storage
with open("./sample-source.txt", "rb") as data:
    blob_client.upload_blob(data)
    print(f"Uploaded sample-source.txt to {blob_client.url}")

    Reference links:

  2. Create an environment variable named AZURE_STORAGE_BLOB_URL:

    set AZURE_STORAGE_BLOB_URL=https://pythonazurestorage12345.blob.core.windows.net

    Replace "pythonazurestorage12345" with the name of your storage account.

    The AZURE_STORAGE_BLOB_URL environment variable is used only by this example. It isn't used by the Azure libraries.

  3. Use the az ad sp create-for-rbac command to create a new service principal for the app. The command creates the app registration for the app at the same time. Give the service principal a name of your choosing.

    az ad sp create-for-rbac --name {service-principal-name}

    The output of this command will look like the following. Make note of these values or keep this window open as you'll need these values in the next step and won't be able to view the password (client secret) value again. You can, however, add a new password later without invalidating the service principal or existing passwords if needed.

    {
  "appId": "00000000-0000-0000-0000-000000000000",
  "displayName": "{service-principal-name}",
  "password": "abcdefghijklmnopqrstuvwxyz",
  "tenant": "11111111-1111-1111-1111-111111111111"
}

    Azure CLI commands can be run in the Azure Cloud Shell or on a workstation with the Azure CLI installed.

  4. Create environment variables for the application service principal:

    Create the following environment variables with the values from the output of the previous command. These variables tell DefaultAzureCredential to use the application service principal.

    • AZURE_CLIENT_ID → The app ID value.
    • AZURE_TENANT_ID → The tenant ID value.
    • AZURE_CLIENT_SECRET → The password/credential generated for the app.
    set AZURE_CLIENT_ID=00000000-0000-0000-0000-000000000000
set AZURE_TENANT_ID=11111111-1111-1111-1111-111111111111
set AZURE_CLIENT_SECRET=abcdefghijklmnopqrstuvwxyz

  5. Attempt to run the code (which fails intentionally):

    python use_blob_auth.py

  6. Observe the error "This request is not authorized to perform this operation using this permission." The error is expected because the local service principal that you're using doesn't yet have permission to access the blob container.

  7. Grant contributor permissions on the blob container to the service principal using the az role assignment create Azure CLI command:

    az role assignment create --assignee <AZURE_CLIENT_ID> \
    --role "Storage Blob Data Contributor" \
    --scope "/subscriptions/<AZURE_SUBSCRIPTION_ID>/resourceGroups/PythonAzureExample-Storage-rg/providers/Microsoft.Storage/storageAccounts/pythonazurestorage12345/blobServices/default/containers/blob-container-01"

    The --assignee argument identifies the service principal. Replace <AZURE_CLIENT_ID> placeholder with the app ID of your service principal.

    The --scope argument identifies where this role assignment applies. In this example, you grant the "Storage Blob Data Contributor" role to the service principal for the container named "blob-container-01".

    • Replace PythonAzureExample-Storage-rg and pythonazurestorage12345 with the resource group that contains your storage account and the exact name of your storage account. Also, adjust the name of the blob container, if necessary. If you use the wrong name, you see the error, "Can not perform requested operation on nested resource. Parent resource 'pythonazurestorage12345' not found."

    • Replace the <AZURE_SUBSCRIPTION_ID> place holder with your Azure subscription ID. (You can run the az account show command and get your subscription ID from the id property in the output.)

    Tip

    If the role assignment command returns an error "No connection adapters were found" when using bash shell, try setting export MSYS_NO_PATHCONV=1 to avoid path translation. For more information, see this issue.

  8. Wait a minute or two for the permissions to propagate, then run the code again to verify that it now works. If you see the permissions error again, wait a little longer, then try the code again.

For more information on role assignments, see How to assign role permissions using the Azure CLI.

4b: Use blob storage with a connection string

  1. Create a Python file named use_blob_conn_string.py with the following code. The comments explain the steps.

    import os
import uuid

# Import the client object from the SDK library
from azure.storage.blob import BlobClient

# Retrieve the connection string from an environment variable. Note that a
# connection string grants all permissions to the caller, making it less
# secure than obtaining a BlobClient object using credentials.
conn_string = os.environ["AZURE_STORAGE_CONNECTION_STRING"]

# Create the client object for the resource identified by the connection
# string, indicating also the blob container and the name of the specific
# blob we want.
blob_client = BlobClient.from_connection_string(
    conn_string,
    container_name="blob-container-01",
    blob_name=f"sample-blob-{str(uuid.uuid4())[0:5]}.txt",
)

# Open a local file and upload its contents to Blob Storage
with open("./sample-source.txt", "rb") as data:
    blob_client.upload_blob(data)
    print(f"Uploaded sample-source.txt to {blob_client.url}")

  2. Create an environment variable named AZURE_STORAGE_CONNECTION_STRING, the value of which is the full connection string for the storage account. (This environment variable is also used by various Azure CLI comments.) You can get the connection string for your storage account by running the az storage account show-connection-string command.

    az storage account show-connection-string --resource-group PythonAzureExample-Storage-rg -name pythonazurestorage12345

    Replace PythonAzureExample-Storage-rg and pythonazurestorage12345 with the resource group that contains your storage account and the exact name of your storage account.

    When you set the environment variable, use the entire value of the connectionString property in the output including the quotes.

  3. Run the code:

    python use_blob_conn_string.py

Again, although this method is simple, a connection string authorizes all operations in a storage account. With production code, it's better to use specific permissions as described in the previous section.

5. Verify blob creation

After running the code of either method, go to the Azure portal, navigate into the blob container to verify that a new blob exists named sample-blob-{random}.txt with the same contents as the sample-source.txt file:

Azure portal page for the blob container, showing the uploaded file

If you created an environment variable named AZURE_STORAGE_CONNECTION_STRING, you can also use the Azure CLI to verify that the blob exists using the az storage blob list command:

az storage blob list --container-name blob-container-01

If you followed the instructions to use blob storage with authentication, you can add the --connection-string parameter to the preceding command with the connection string for your storage account. To learn how to get the connection string, see the instructions in 4b: Use blob storage with a connection string. Use the whole connection string including the quotes.

6: Clean up resources

Run the az group delete command if you don't need to keep the resource group and storage resources used in this example. Resource groups don't incur any ongoing charges in your subscription, but resources, like storage accounts, in the resource group might incur charges. It's a good practice to clean up any group that you aren't actively using. The --no-wait argument allows the command to return immediately instead of waiting for the operation to finish.

az group delete -n PythonAzureExample-Storage-rg  --no-wait

You can also use the ResourceManagementClient.resource_groups.begin_delete method to delete a resource group from code. The code in Example: Create a resource group demonstrates usage.

If you followed the instructions to use blob storage with authentication, it's a good idea to delete the application service principal you created. You can use the az ad app delete command. Replace the <AZURE_CLIENT_ID> placeholder with the app ID of your service principal.

az ad app delete --id <AZURE_CLIENT_ID>

See also