Issue in uploading the CSV (not json content) file in one drive location using graph API in ADF

Question

I'm copying a CSV file from Azure Blob Storage to OneDrive using the Graph API in Azure Data Factory (ADF).

I'm using a Copy Activity to move the file from a Blob dataset (source) to OneDrive (sink) via a REST service. While the file uploads to OneDrive with a .csv extension, the actual content inside is in JSON format. However, I need the content to remain in CSV format. How can I achieve this?

Answer 1

Hi Thirumurugan Rajan,

Thanks for reaching out to Microsoft Q&A.

1. Change the Copy Behavior:

Ensure that the source dataset in ADF is set to use DelimitedText (CSV) format.

Check the sink settings in your Copy Activity to make sure you're not serializing the content to JSON. You may need to adjust the mappings and settings to avoid any JSON serialization.

2. Modify the REST API Request:

In ADF, when uploading the file via REST API, ensure that you're sending the file as raw binary content rather than converting it to JSON.

To do this:

Set the HTTP method to 'PUT' or 'PATCH' (as appropriate for your OneDrive upload API call).

Specify the correct headers, particularly the 'Content-Type' as 'text/csv', so that OneDrive treats the file content as CSV.

Example of correct headers:

'''json

{

"Content-Type": "text/csv"

}

'''

3. Using an Azure Function for Upload:

If the ADF Copy Activity continues to serialize the content to JSON, consider creating an Azure Function that directly uploads the CSV file to OneDrive via the Graph API.

The function can handle the file as a stream or as binary content and upload it using the correct MIME type ('text/csv').

4. Using ADF Web Activity:

If the Copy Activity doesn't meet your requirements, you could use an ADF Web Activity that triggers a REST API call directly to OneDrive with the correct payload and headers. This way, you control the file content and format during the transfer.

Example REST API Call (Graph API to upload file):

'''http

PUT https://graph.microsoft.com/v1.0/me/drive/root:/path/to/yourfile.csv:/content

Headers:

Content-Type: text/csv

Body:

<Raw CSV content>

'''

This should ensure that the file content remains in CSV format when uploaded to OneDrive.

---

Please 'Upvote'(Thumbs-up) and 'Accept' as an answer if the reply was helpful. This will benefit other community members who face the same issue.