TypeError: function() argument 'code' must be code, not str in Azure Synapse Notebook

Question

I am having below data frame and wanted to save the data frame as a CSV file in the Azure Data lake. My data frame is called 'df'. I am using Azure Synpase Notebook

df.to_csv('abfss://email address removed for privacy reasons/Jobs_newdata/data.csv', sep=',', encoding='utf-8', index=False)  

Getting below error message when I tried to run the above code

Can anyone advise what is the issue here?

Answer 1

Hello @Dhilip Subramanian ,
Thanks for the question and using MS Q&A platform.

As we understand the ask here is to get write the dataframe to a storage account in the CSV format, please do let us know if its not accurate.
You are calling the storage account as
abfss://email address removed for privacy reasons/Jobs_newdata/data.csv
You said call this
abfss://yourcontainername@YourStorageAccountName.dfs.core.windows.net/email address removed for privacy reasons/Jobs_newdata/data.csv

The below piece of code just works fine .

filePath = "abfss://yourcontainername@analyticssynapsews.dfs.core.windows.net/email address removed for privacy reasons/Jobs_newdata/data.csv"

d = {'col1': [1, 2], 'col2': [3, 4]}
pd = pandas.DataFrame(d)
df = spark.createDataFrame(pd)
df.write.format("csv").mode("overwrite").save(filePath)

Please do let me if you have any queries.
Thanks
Himanshu

---

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Answer 2

Dear @HimanshuSinha-msft ,

Is there way to save to csv without using spark ?, because it divides the data.