Read pickle file from Azure File Share using python code

Satish Barnala 11 Reputation points Microsoft Vendor
2021-07-14T12:32:53.873+00:00

I have pickle files in Azure Storage Account under File Share. I want to read them programmatically from a python code.

I'm trying the below code snippet, but it doesn't work. Please help!

import pandas as pnd
from azure.storage.fileshare import ShareFileClient
from azure.storage.fileshare import ShareServiceClient

connection_string = "XXXXXXX"
service = ShareServiceClient.from_connection_string(conn_str=connection_string)
file_client = ShareFileClient.from_connection_string(conn_str=connection_string, share_name="YYYYYY", file_path="ZZZZZZ")

with open("file1.pkl", "wb") as file_handle:
data = file_client.download_file()
data.readinto(file_handle)
dfDataContent = pnd.read_pickle(data)

Azure Files
Azure Files
An Azure service that offers file shares in the cloud.
1,152 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. shiva patpi 13,131 Reputation points Microsoft Employee
    2021-07-15T00:26:33.59+00:00

    Hello @Satish Barnala ,
    Thanks for your query !
    Can we know what exact error message you are getting by running the above piece of code ?
    Were you able to download the pkl file locally by uncommenting the last line ?

    I just tried your code , it has downloaded the pkl file successfully but in the last line while reading the file got an error w.r.t file handler etc.

    Probably you are getting error in the last line after downloading while reading the .pkl file ?

    In general , pkl file should have the right format before reading out the contents .
    You might want to save it in a proper format using dataframe function and then use read_pickle ?

    Couple of related articles:

    https://datascienceparichay.com/article/read-pickle-file-as-pandas-dataframe/#:~:text=You%20can%20use%20the%20pandas%20read_pickle%20%28%29%20function,file%20storing%20the%20data%20you%20want%20to%20read.

    read_Pickle documentation:
    https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_pickle.html

    Try to give the relative path in read_pickle function as mentioned in the above article.

    Let us know if the above document helps you out in resolving the issue.