Write csv file to Sharepoint from databricks notebook using scala

Question

We want to write csv file in sharepoint using scala from databricks notebook but I haven't found solution online or in community. Is there's a way to do it ? Is it possible from notebook ?If not, is there any alternative way to do it ?

Answer 1

Hi @ak , welcome to Q&A forum!

As mentioned in this thread: Reading data from Sharepoint Using Databricks

Reading data from sharepoint using Azure databricks is not possible. Sharepoint is not supported source in Azure databricks.

For more details, kindly refer to: Data sources

---

If the answer is helpful, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Answer 2

There are two way to do it.

@Elsie Lu_MSFT

1. We cannot save file in Sharepoint directly though Databricks but We can upload file using Graph API which will support uploading large file.
   I am thinking of creating jar file and call the function through spark scala.

https://devzigma.com/java/upload-files-to-sharepoint-using-java/

2. I wrote code in C# to upload large file in sharepoint using Microsoft.SharePoint.Client . I am checking if I can convert it to UDF function and call it from spark.

https://learn.microsoft.com/en-us/sharepoint/dev/solution-guidance/upload-large-files-sample-app-for-sharepoint
https://learn.microsoft.com/en-us/dotnet/spark/how-to-guides/udf-guide#:~:text=NET%20for%20Apache%20Spark.,logic%20defined%20in%20the%20UDF.

I wish we have straight forward solution but it seems possible . Lot of work need to be done to make it work.