question

JanBoldt-5975 avatar image
0 Votes"
JanBoldt-5975 asked KranthiPakala-MSFT commented

The sharepoint list is empty when import from Azure Data factory, but I can only see specific coloumns and import them, not the full list

I have followed the guideline to import a sharepoint list using ADF:

https://docs.microsoft.com/en-us/azure/data-factory/connector-sharepoint-online-list?tabs=data-factory

I have created the Linked service. When I need to define the source in the pipeline, then when selecting the list, then no data is shown in the PreviewData. I can select a column which is also shown and see the data. I do not know what it is that I have done wrong, is there some setting on the list that needs to be specified.


201984-image.png


azure-data-factoryoffice-sharepoint-online
image.png (12.1 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

KranthiPakala-MSFT avatar image
0 Votes"
KranthiPakala-MSFT answered JanBoldt-5975 commented

Hello @JanBoldt-5975,

Thanks for the question and using MS Q&A platform.

I tried to reproduce the issue on my end but no luck. I was able to see the data if I do a $select=* query on the selected List

202513-image.png

Below is my dataset configuration, please make sure to select the List name in your dataset settings and try to do a preview data from the data set and then try to use the query in Copy Data activity source settings. If you are able to see the data from dataset preview tab then you should be able to see the data using Query under Copy Activity Source preview section.

202464-image.png

If you still notice this issue, to work around, I would suggest to select the Table option instead of Query under Copy activity Source settings as you are trying to view the whole list data. If you want to filter specific data related to your list then use Query option.

202474-image.png

Hope this will help. Please let us know if any further queries.


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification

  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators



image.png (59.8 KiB)
image.png (19.6 KiB)
image.png (18.7 KiB)
· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I even tried to remove the complete query and did a preview and was able to view the data.


202552-image.png

I would suggest to preview the data from your dataset and check if you can view from there.


0 Votes 0 ·
image.png (59.1 KiB)

Thank you for your reply. I have tried all possible ways (table and query) of extracting the data without luck. What is included in the "GetBearerToke" and would you have a link to a guide on how to set it up. I cannot see that as included in the guide I have used:

https://docs.microsoft.com/en-us/azure/data-factory/connector-sharepoint-online-list?tabs=data-factory

Could you provide a link to the guide you are using.

I am not getting any data when I preview the dataset of the sharepoint list, but when I choose one of the columns which is available, then data is shown.



0 Votes 0 ·

202816-picture1.jpg


The picture above shows that there is no data in the list.

202777-picture3.png


The picture above shows that there is data in one of the columns.

202787-picture4.png



The picture above shows the columns in the list.

0 Votes 0 ·
picture1.jpg (13.6 KiB)
picture3.png (6.1 KiB)
picture4.png (38.2 KiB)
JanBoldt-5975 avatar image
1 Vote"
JanBoldt-5975 answered KranthiPakala-MSFT commented

I found the mistake. I had replaced the address in the Scope="..." with the direct reference to the sharepoint site. It should be as specified as below.

<AppPermissionRequests AllowAppOnlyPolicy="true">
<AppPermissionRequest Scope="http://sharepoint/content/sitecollection/web" Right="Read"/>
</AppPermissionRequests>

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @JanBoldt-5975,

Glad to know that you were able to find the actual root cause and appreciate much for sharing your findings with us here as it can be helpful for others who read this thread.

Thank you and have a good day!

0 Votes 0 ·