Azure Data Factory Sharepoint Online List connector can't preview data

Question

Azure Data Factory Sharepoint Online List connector can't preview data

dazhi bai 0

Hi, thanks in advance;

I am using "sharepoint online connector" in Azure Datafactory to copy data from SPO list to a sql db; below is the issue,

1, i can connect to the spo list site, see below pic2;

2, but i can't preview data in dataset of the list;

User's image

Smaran Thoomu 175 Reputation points Microsoft Vendor

2023-09-07T11:32:22.2533333+00:00
@dazhi bai Welcome to the Microsoft Q&A platform and thank you for posting your question here.

I understand that you are encountering an issue while using the SharePoint Online connector in Azure Data Factory to copy data from a SharePoint Online list to a SQL database. Specifically, you are unable to preview data in the dataset of the list. I'll be delighted to assist you with this.

To troubleshoot the issue you are facing, please consider the following steps:

Try running the pipeline in debug mode. This allows you to view the input and output data for each activity in the pipeline, which can help identify any data flow issues and troubleshoot errors.

Ensure that the account used to connect to SharePoint and the SQL database has the necessary permissions to access the data. This account should have at least read access to the SharePoint list and write access to the SQL database.

Verify that the list schema is valid and matches the schema defined in the Azure Data Factory pipeline. If the schemas don't align, you may encounter issues while previewing the data.

Please be sure to check the data size when using the preview feature, as it has a limitation of either 1000 rows or 4 MB of data, whichever is reached first. If the data size surpasses this limit, you won't be able to preview it.

Additionally, could you please provide more details about the dataset you're attempting to preview? This will help us provide you with more targeted assistance in resolving the issue.

For more information on troubleshooting issues with the SharePoint Online connector in ADF, you can refer to the following documentation:

Troubleshoot the SharePoint Online connector in Azure Data Factory

I hope this helps! Let me know if you have any further questions.
Smaran Thoomu 175 Reputation points Microsoft Vendor

2023-09-08T12:31:45.2933333+00:00

@dazhi bai We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Victory Bai 0 Reputation points

2023-09-08T13:21:16.7133333+00:00

@Smaran Thoomu , thanks for the response;

pls see below facts,

1, the linked service test connection is successful, and the dataset can retrieve the spo site lists, so i suppose the permissions are right in both spo and adf;

2, but I can't preview data for any list, i tried some list only have 50 rows, still can't preview data from it;
Smaran Thoomu 175 Reputation points Microsoft Vendor

2023-09-11T13:06:02.0666667+00:00
Hi @dazhi bai
Please review the following steps:

Make sure that the data preview settings in the dataset configuration are correct and that they match the format of the SPO list. You can specify the data preview settings by selecting the "Preview data" tab in the dataset configuration. You can refer to the Azure Data Factory documentation for more information.

Ensure that the column mapping and data format specified in the dataset configuration are correct and match the columns in the SPO list. If the column mapping or data formats are incorrect, you may not be able to preview the data.

If you have checked all these factors and are still unable to preview data in ADF, you may open a support ticket for further assistance.
Victory Bai 0 Reputation points

2023-09-12T01:54:22.1033333+00:00

got it resolved by reconfigure spo site permission scope, thanks.
Smaran Thoomu 175 Reputation points Microsoft Vendor

2023-09-13T04:52:49.1833333+00:00

@dazhi bai - Thanks for the question and using MS Q&A platform.

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.
Smaran Thoomu 175 Reputation points Microsoft Vendor

2023-09-14T10:01:55.2733333+00:00

@dazhi bai Following up to see if the below answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Smaran Thoomu 175 Reputation points Microsoft Vendor

2023-09-07T11:32:22.2533333+00:00

@dazhi bai Welcome to the Microsoft Q&A platform and thank you for posting your question here.

I understand that you are encountering an issue while using the SharePoint Online connector in Azure Data Factory to copy data from a SharePoint Online list to a SQL database. Specifically, you are unable to preview data in the dataset of the list. I'll be delighted to assist you with this.

To troubleshoot the issue you are facing, please consider the following steps:

Try running the pipeline in debug mode. This allows you to view the input and output data for each activity in the pipeline, which can help identify any data flow issues and troubleshoot errors.

Ensure that the account used to connect to SharePoint and the SQL database has the necessary permissions to access the data. This account should have at least read access to the SharePoint list and write access to the SQL database.

Verify that the list schema is valid and matches the schema defined in the Azure Data Factory pipeline. If the schemas don't align, you may encounter issues while previewing the data.

Please be sure to check the data size when using the preview feature, as it has a limitation of either 1000 rows or 4 MB of data, whichever is reached first. If the data size surpasses this limit, you won't be able to preview it.

Additionally, could you please provide more details about the dataset you're attempting to preview? This will help us provide you with more targeted assistance in resolving the issue.

For more information on troubleshooting issues with the SharePoint Online connector in ADF, you can refer to the following documentation:

Troubleshoot the SharePoint Online connector in Azure Data Factory

I hope this helps! Let me know if you have any further questions.
Smaran Thoomu 175 Reputation points Microsoft Vendor

2023-09-08T12:31:45.2933333+00:00

@dazhi bai We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Victory Bai 0 Reputation points

2023-09-08T13:21:16.7133333+00:00

@Smaran Thoomu , thanks for the response;

pls see below facts,

1, the linked service test connection is successful, and the dataset can retrieve the spo site lists, so i suppose the permissions are right in both spo and adf;

2, but I can't preview data for any list, i tried some list only have 50 rows, still can't preview data from it;
Smaran Thoomu 175 Reputation points Microsoft Vendor

2023-09-11T13:06:02.0666667+00:00

Hi @dazhi bai
Please review the following steps:

Make sure that the data preview settings in the dataset configuration are correct and that they match the format of the SPO list. You can specify the data preview settings by selecting the "Preview data" tab in the dataset configuration. You can refer to the Azure Data Factory documentation for more information.

Ensure that the column mapping and data format specified in the dataset configuration are correct and match the columns in the SPO list. If the column mapping or data formats are incorrect, you may not be able to preview the data.

If you have checked all these factors and are still unable to preview data in ADF, you may open a support ticket for further assistance.
Victory Bai 0 Reputation points

2023-09-12T01:54:22.1033333+00:00

got it resolved by reconfigure spo site permission scope, thanks.
Smaran Thoomu 175 Reputation points Microsoft Vendor

2023-09-13T04:52:49.1833333+00:00

@dazhi bai - Thanks for the question and using MS Q&A platform.

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.
Smaran Thoomu 175 Reputation points Microsoft Vendor

2023-09-14T10:01:55.2733333+00:00

@dazhi bai Following up to see if the below answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 1

@dazhi bai - Thanks for the question and using MS Q&A platform.

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer .

Issue: Azure Data Factory Sharepoint Online List connector can't preview data
Solution: Got it resolved by reconfigure spo site permission scope.
Ensure that the account used to connect to SharePoint and the SQL database has the necessary permissions to access the data. This account should have at least read access to the SharePoint list and write access to the SQL database.

If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

I hope this helps!

If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.

Azure Data Factory Sharepoint Online List connector can't preview data

1 answer

Activity