I am trying to connect to a specific container in my Azure Storage Account and read the files. My goal is to list all the blobs in the container, download each blob into an in-memory byte stream, save the stream contents to a local Parquet file, and then read the Parquet file into a pandas DataFrame.
Here is the code I am using:
from azure.storage.blob import BlobServiceClient, ContainerClient
from io import BytesIO
import pandas as pd
container_name = 'poc-powerapp-container'
sas_token = 'your_sas_token'
blob_service_url = 'your_blob_service_url'
blob_service_client = BlobServiceClient(account_url=blob_service_url, credential=sas_token)
container_client = blob_service_client.get_container_client(container_name)
blobs_list = container_client.list_blobs()
for blob in blobs_list:
if 'parquet' in blob.name.lower():
try:
blob_client = blob_service_client.get_blob_client(container=container_name, blob=blob.name)
stream = BytesIO()
blob_client.download_blob().readinto(stream)
stream.seek(0)
local_parquet_file_path = "local_file_path.parquet"
with open(local_parquet_file_path, 'wb') as f:
f.write(stream.getbuffer())
print(f"Parquet file saved as '{local_parquet_file_path}'")
df = pd.read_parquet(local_parquet_file_path)
print(df.head())
except Exception as e:
print(f"Error processing blob {blob.name}: {e}")
However, I am encountering the following error:
HttpResponseError: The requested URI does not represent any resource on the server.
HttpResponseError Traceback (most recent call last)
...
File ~\AppData\Roaming\Python\Python312\site-packages\azure\storage\blob\_list_blobs_helper.py:96, in BlobPropertiesPaged._get_next_cb(self, continuation_token)
89 return self._command(
90 prefix=self.prefix,
91 marker=continuation_token or None,
92 maxresults=self.results_per_page,
93 cls=return_context_and_deserialized,
94 use_location=self.location_mode)
95 except HttpResponseError as error:
---> 96 process_storage_error(error)
What I've Tried
- Verified the SAS token and Blob service URL.
- Checked the permissions for the storage account and container.
- Ensured that the blobs exist in the container.
Questions
- What could be causing the
HttpResponseError: The requested URI does not represent any resource on the server error?
- Is there something wrong with the way I am listing or accessing the blobs?
- Are there any additional steps I should take to debug this issue?
Any help or guidance would be greatly appreciated!
Thank you!