Generating SAS Token for Azure Datalake does not work properly

Oweys 0 Reputation points

I'm having trouble accessing my Azure Data Lake container using a SAS token generated by Python. While a manually generated SAS token works, the Python-generated one leads to an authentication error (see picture):.User's image

Now this is not the best solution, so I tried generating the token based on my Accountname, accountkey and the container with python. I also set the starting point 5 minutes prior to the current timestamp. Here is the code:

class ADLImporter:
    def __init__(self):
        self.logger_inst = logger.create_logger("INFO")
        sas_token = self._generate_sas_token(account_name=ADL.ACCOUNTNAME, 
        # TODO remove sas token from env and check why it still does not work.
        sas_url = f"{ADL.ACCOUNTNAME}/{ADL.CONTAINER_NAME}?{sas_token}"
        self.container_client = ContainerClient.from_container_url(sas_url)"Connected to container.")

    def _generate_sas_token(self, account_name, account_key, container_name, expiry_time, start_time):
        Generate a SAS token for a blob container.

          account_name (str): The storage account name.
          account_key (str): The storage account key.
          container_name (str): The name of the blob container.
          expiry_time (datetime): The time at which the SAS token expires.

          str: The SAS token.
        sas_token = generate_container_sas(
            permission=ContainerSasPermissions(read=True, list=True),
        )  "Created SAS Token.")

        return sas_token

However, if I replace the manually generated token with the one I generate with python, I get an authorization error:

......exec("raise error from None") # pylint: disable=exec-used # nosec File "<string>", line 1, in <module> azure.core.exceptions.ClientAuthenticationError: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. Content: <?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. RequestId:2b4c23f5-601e-002d-41b0-112e45000000 Time:2023-11-07T19:30:45.5693910Z</Message><AuthenticationErrorDetail>Signature did not match. String to sign used was rl....

I do not know what I am doing wrong. At the end I end up with these tokens:

manually: sp=rl&st=2023-11-07T20:00:00Z&se=2023-11-08T03:00:00Z&spr=https&sv=2022-11-02&sr=c&sig=<sig...>

with python: se=2023-11-08T00%3A26%3A51Z&sp=rl&spr=https&sv=2023-08-03&sr=c&sig=<sig...>

Any ideas what could be wrong?

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,195 questions
{count} votes

1 answer

Sort by: Most helpful
  1. PRADEEPCHEEKATLA-MSFT 70,271 Reputation points Microsoft Employee

    @Oweys - Thanks for the question and uisng MS Q&A platform.

    The issue you are facing seems to be related to the format of the SAS token that you are generating with Python. The SAS token format is crucial, and even a small difference in formatting can lead to authentication errors. Let's go through your code and identify the issue.

    Here's the manually generated SAS token for reference:


    And here's the Python-generated SAS token:


    The differences between the two tokens are the date/time format, the version (sv), and possibly other factors. Let's address these issues one by one.

    Date/Time Format: The date and time format in the manually generated token is in a specific ISO 8601 format. In the Python-generated token, you are encoding the time with URL encoding (e.g., %3A for :). You should format the datetime correctly in ISO 8601 format.

    Version (sv): The manually generated token specifies sv=2022-11-02, which is the SAS token version. In your Python code, you're using sv=2023-08-03. Make sure to use the correct SAS token version. You should use the same version as the one that works, which seems to be sv=2022-11-02.

    Check Other Parameters: Make sure that all other parameters, such as sp (permissions), spr (protocol), and sr (resource type), are consistent with the manually generated token.

    Here's how you can modify your _generate_sas_token function to ensure the correct formatting:

    from import generate_container_sas, ContainerSasPermissions
    def _generate_sas_token(self, account_name, account_key, container_name, expiry_time, start_time):
        permissions = ContainerSasPermissions(read=True, list=True)
        sas_token = generate_container_sas(
            version='2022-11-02'  # Use the correct version
   "Created SAS Token.")
        return sas_token

    Ensure that you pass the correct version ('2022-11-02') as shown above and format the start_time and expiry_time as ISO 8601 datetime strings.

    After making these changes, your Python-generated SAS token should match the manually generated one, and it should work correctly for accessing your Azure Data Lake container.

    I hope this information helps. If you have any further questions or need more clarification, please let me know.

    0 comments No comments