Edit

Share via

Configure SharePoint Online List in a Copy activity

  • Article

This article outlines how to use the copy activity in a data pipeline to copy data from SharePoint Online List.

Prerequisites

  1. The SharePoint List Online connector uses service principal authentication to connect to SharePoint. Follow these steps to set it up:

  2. On your SharePoint tenant, disable DisableCustomAppAuthentication through PowerShell.

    Install-Module -Name Microsoft.Online.SharePoint.PowerShell`  
Connect-SPOService -Url https://<your-domain>-admin.sharepoint.com`  
Set-SPOTenant -DisableCustomAppAuthentication $false`  
Disconnect-SPOService

  3. Register an application with the Microsoft identity platform. To learn how, go to Quickstart: Register an application with the Microsoft identity platform. Make note of these values, which you use to define the connection:

    • Application ID
    • Application key
    • Tenant ID

  4. Use the following steps to grant SharePoint Online site permission to your registered application. To grant permission, you need a site admin role.

    1. Open a SharePoint Online site link, for example https://[your_site_url]/_layouts/15/appinv.aspx (replace the site URL).

    2. Search the application ID you registered, fill the empty fields, and then select Create.

      • App Domain: contoso.com

      • Redirect URL: https://www.contoso.com

      • Permission Request XML:

        <AppPermissionRequests AllowAppOnlyPolicy="true">
    <AppPermissionRequest Scope="http://sharepoint/content/sitecollection/web" Right="Read"/>
</AppPermissionRequests>

      Screenshot showing request XML.

      Note

      In the context of configuring the SharePoint connector, the App Domain and Redirect URL refer to the SharePoint app that you've registered in Microsoft Entra ID to allow access to your SharePoint data. The App Domain is the domain where your SharePoint site is hosted. For example, if your SharePoint site is located at https://contoso.sharepoint.com, then the App Domain would be contoso.sharepoint.com. The Redirect URL is the URL that the SharePoint app redirects to after the user has authenticated and granted permissions to the app. This URL should be a page on your SharePoint site that the app has permission to access. For example, you could use the URL of a page that displays a list of files in a library, or a page that displays the contents of a document.

    3. Select Trust It for this app.

Supported configuration

For the configuration of each tab under a copy activity, go to the following sections respectively.

General

For General tab configuration, go to General.

Source

The following properties are supported for SharePoint Online List under the Source tab of a copy activity.

Screenshot showing source tab and the list of properties.

The following properties are required:

  • Data store type: Select External.

  • Connection: Select a SharePoint Online List connection from the connection list. If no connection exists, then create a new HTTP connection by selecting New.

  • Use query: Choose either List name or Query as your use query. The configuration of each setting is:

    • List name: The name of the SharePoint Online list.

      Screenshot showing list name.

    • Query: The OData query to filter the data in SharePoint Online list. For example, "$top=1".

      Screenshot showing query.

Under Advanced, you can specify the following fields:

  • Request timeout: The wait time to get a response from SharePoint Online. Default value is 5 minutes (00:05:00).

  • Additional columns: Add additional data columns to store source files' relative path or static value. Expression is supported for the latter.

    Screenshot showing additional columns.

Mapping

For Mapping tab configuration, go to Configure your mappings under mapping tab.

Settings

For Settings tab configuration, go to Configure your other settings under settings tab.

Table summary

The following table contains more information about a copy activity in SharePoint Online List.

Source

Name Description Value Required JSON script property
Data store type Your data store type. External Yes /
Connection Your connection to the source data store. <your connection> Yes connection
Use query You can choose List name or Query as your use query. -List name
-Query		 No type
List name The name of the SharePoint Online list. <your connection> Yes listName
Request timeout The wait time to get a response from SharePoint Online. Default value is 5 minutes (00:05:00). timespan No requestTimeout
Additional columns Add additional data columns to store source files' relative path or static value. Expression is supported for the latter. • Name
• Value		 No additionalColumns:
• name
• value

Related content