Refresh Access Token in Azure Synapse Copy Data Activity

Abhinav Gundapaneni 56 Reputation points Microsoft Employee

I have a long running Azure Synapse pipeline which fetches the access token through an Auth API call and uses this for the Copy Data Activity that loads millions of records from an external API to ADLS. The Copy data activity is failing after 30 minutes which is the expiry period for the access token. We tried to use Until activity to refresh the token before 30 minutes but the copy activity doesn’t pick the new value from the variable after the activity starts.

Since this is an external API call, we couldn’t use service principal authentication. We tried OAuth2 Authentication in the linked service for our API but the access token type returned was “bearer” instead of “Bearer” and the pipeline gives an error saying access token type is invalid. On a side note, Does the copy activity refreshes the token on its own if we use OAuth2 Authentication?

Also, is there any elegant and efficient way to handle token refreshes for a long running copy data activity?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,565 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,928 questions
{count} vote