question

Adrian-5124 avatar image
0 Votes"
Adrian-5124 asked vaishnavav99 edited

Python Code upload csv(df) back to azure blob

Hey

I've got some code that downloads a blob as xlsx from azure blob storage, it loads it into a data frame and runs some transformation on it.

I would now like to save this df back into azure blob as a csv, preferably replacing the original file. This is the code I currently have. I am able to download the blob, run the transformations and save it locally, I would like to now push it to blob and preferable replace the original one.

Does anyone know of an efficient way of doing so? I've been having some troubles incorporating the information I found online into my code.

Thanks!


 import pandas as pd
 import numpy as np
 import datetime
 import os, uuid
 from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient, __version__
    
 def function (df):
     (df operations)
     a = df.to_csv('current19.csv', index=False)


 CONNECTION_STRING = ""
 CONTAINERNAME = ""
 BLOBNAME = ""
 LOCALFILENAME = "" 
    
 blob_service_client = BlobServiceClient.from_connection_string(CONNECTION_STRING) #instantiate new blobservice with connection string
 container_client = blob_service_client.get_container_client(CONTAINERNAME) #instantiate new containerclient
 blob_client = blob_service_client.get_blob_client(container = CONTAINERNAME, blob=BLOBNAME)
    
 #READ PRODUCTS FILE
 f = open(LOCALFILENAME, "wb")
 f.write(blob_client.download_blob().content_as_bytes())
 f.close()
 df = pd.read_excel(r''+LOCALFILENAME)
 #print(df)


 function(df)




This currently download blob and saves it locally, I need to add a few lines so it pushed it back to azure blob and overrides the original file


azure-logic-appsazure-blob-storage
· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Just wanted to let you know that I am looking into this and will get back with you shortly. You might also look into using Azure Data Factory as it now support Excel formats.


2 Votes 2 ·

Hey, I actually managed to do it, and switched to saving it as an xlsx file.

The next step would be to actually automate this, as the moment I am stating the filename to be downloaded. Do you know of a way for this code to automatically download new files that will be saved in my blob storage ? Instead of passing the actual name of the new file in code.

Thanks!

0 Votes 0 ·

How were you able to figure it out? Mind sharing the details?

0 Votes 0 ·

you can read the csv as a data frame and try upload it using azure.storage.blob import BlobClient

 blob_client = BlobClient.from_connection_string(storage_connection_string,container_name,filename+".csv")
 blob_client.upload_blob(df)


0 Votes 0 ·

0 Answers