Data Factory does not export sharepoint list completely

Raphael Jöbges 21 Reputation points
2022-06-15T13:47:46.26+00:00

Hello

I am trying to export a sharepoint list to a parquet file in my datalake. This worked well so far but now I am detecting some tables which have more columns in sharepoint than in the Data Factory Export. I can see the missing columns as seperate tables in the Data Factory Dataset that connects to Sharepoint, like Lookup Tables. I assume that there are seperate Sharepoint Lists for the relevant columns so that one can choose from a list if a new dataset is added in Sharepoint. Unfortunately there is no way for me to match the single lists to the whole dataset because the lists have only one column from which to choose the entries in Sharepoint.
I am not sure if this is a problem on Sharepoint or on the Data Factory side. Is there any way to extract the tables exactly the way I can see them in Sharepoint?

Thanks
Raphael

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,209 questions
SharePoint
SharePoint
A group of Microsoft Products and technologies used for sharing and managing content, knowledge, and applications.
11,175 questions
{count} votes

Accepted answer
  1. KranthiPakala-MSFT 46,602 Reputation points Microsoft Employee
    2022-06-27T22:43:22.987+00:00

    Hello @Raphael Jöbges ,

    Thanks for additional clarification regarding the issue.
    Yes, if your missing column is complex type (Multiple choice type) then it is not supported by the ADF SP Online list connector which is why you are seeing this issue.

    Please refer to below note from public doc:

    215489-image.png

    Ref doc: https://learn.microsoft.com/en-us/azure/data-factory/connector-sharepoint-online-list?tabs=data-factory#data-type-mapping-for-sharepoint-online-list

    Please feel free to log your feedback in IDEAS forum : https://feedback.azure.com/d365community/forum/1219ec2d-6c26-ec11-b6e6-000d3a4f032c.
    All the feedback shared in this forum are actively monitored and reviewed by respective product owners. Also please do share the feedback link once it is posted so that we can share the info with respective product team for further review.

    Hope this info helps.


1 additional answer

Sort by: Most helpful
  1. Raphael Jöbges 21 Reputation points
    2022-07-08T05:50:38.677+00:00

    Hello @KranthiPakala-MSFT

    thanks for clearification.

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.