Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,429 questions
How to fix invalid access token issue for resources connected to ADF via system managed identity?
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 57.99999999999999%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESA%3C/text%3E%3C/svg%3E)
Saif Ahmed
0
Reputation points
We're facing an intermittent error with Azure Data Factory's 'Copy Data' feature, and we're unsure of the cause. Please note that we have also reached out to Databricks regarding this and was referred to MS/Azure support.
The task is straightforward: use Azure Data Factory to transfer data from a specific catalog and table in Databricks Unity Catalog to a table in Azure Synapse. We've set up all connections accurately, but during testing, the process occasionally runs successfully and other times fails with an error. Below is a screenshot showing the error:
Interestingly, the process sometimes runs without any issues. The screenshot below shows some of the copy activities are successful upon multiple retries:
Additional Context:
- This process executes nightly in another ADF pipeline without any errors. The issue arises only in the new ADF setup, despite using the same connection pattern and the same setup.
- The problem isn't specific to a particular catalog; we encounter the same error across different catalogs.
Do you have any insights into the activity's instability or potential resolutions? Your expertise would be greatly appreciated.
{count} votes
-
Nandan Hegde 34,866 Reputation points MVP2025-03-28T16:50:10.1433333+00:00 Based on my understanding, the access token that was created and used might have expired thereby creating this issue. This seems to be more of an Entra aspect w.r.t token expiration time would be my 1st thought
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 99%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJN%3C/text%3E%3C/svg%3E)
J N S S Kasyap 1,520 Reputation points Microsoft External Staff2025-03-28T17:00:25.9833333+00:00 Hi @Saif Ahmed
The issue you're encountering with Azure Data Factory (ADF) involves an intermittent "invalid access token" error when using a system-managed identity to connect to resources like Databricks Unity Catalog and Azure Synapse.
Let's understand the potential issues and solutions
The error "invalid access token" occurs intermittently because the system-managed identity's token might be expiring during the Copy activity, or ADF is failing to refresh it properly. This can happen due to network delays, token validation issues in Databricks Unity Catalog, or regional Azure AD service hiccups.
The ADF system-managed identity likely lacks proper permissions in Databricks Unity Catalog. It needs USAGE on the catalog and schema, and SELECT on the specific table (e.g., spark_catalog.default.your_table). Similarly, ensure the identity has Contributor or Storage Blob Data Contributor roles in Synapse and the underlying storage account.
Since the nightly pipeline works fine, there might be a configuration difference in the new ADF setup. Check for differences in linked services, integration runtime (IR), or pipeline settings (e.g., concurrency, timeout). Ensure the new ADF uses the same IR and region as the working pipeline.
Databricks Unity Catalog has strict access controls. The system-managed identity must be explicitly granted permissions at the catalog, schema, and table levels. Without these, the token will be rejected, leading to the AnalysisException error seen in the logs.
To mitigate intermittent failures, add retry logic to the Copy activity (e.g., 3 retries with a 30-second interval). Also, test the pipeline with a smaller dataset to see if the issue is related to activity duration or throttling by Databricks/Synapse.
I hope this information helps. Please do let us know if you have any further queries.
Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.
Thank you.
-
J N S S Kasyap 1,520 Reputation points Microsoft External Staff
2025-03-31T00:45:55.06+00:00 @Saif Ahmed
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help. -
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 57.00000000000001%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERG%3C/text%3E%3C/svg%3E)
Rakesh Govindula 965 Reputation points Microsoft External Staff2025-03-31T04:54:57.8566667+00:00 Please try the below debugging steps:
- Check ADF Logs & Databricks Logs
- Look at the detailed error logs in ADF Monitor.
- In Databricks, check
Cluster LogsandAudit Logsfor failed token authentication attempts.
- Manually Validate Token Expiry
- If using Managed Identity, try fetching a token manually:
az account get-access-token --resource https://databricks.azure.net - If it expires too quickly, you may need to adjust ADF’s token retrieval method.
- If using Managed Identity, try fetching a token manually:
- Run the Same Query Manually in Databricks
- Open a Databricks notebook and try running the same Unity Catalog table query.
- If it fails there too, the issue is likely with token access or permissions.
If you are still facing the issue, please provide the linked service configurations along with the detailed error message.
Hope this helps. Do let us know if you any further queries.
- Check ADF Logs & Databricks Logs
-
Rakesh Govindula 965 Reputation points Microsoft External Staff
2025-04-01T03:32:47.64+00:00 @Saif Ahmed We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
-
AnnuKumari-MSFT 34,451 Reputation points Microsoft Employee2025-04-07T10:50:47.74+00:00 We haven't heard back from you on the above query. Just checking if you got a chance to see the suggested responses. Kindly let us know if the above response was helpful. Thanks
Sign in to comment
Sign in to answer