Why are my Data Factory pipelines failing ocassionally due to same error: "request_throttled, Message: Your app has been throttled by AAD due to too many requests"?

61521973 31 Reputation points
2022-09-28T06:05:03.99+00:00

Hi!

I use Data Factory for multiple projects (different tenants) that have been running daily for months, some of them for over one year.
Along my pipelines, some linked-services are configured by getting secrets from Key vaults resided in same resource groups as data factory.

In the past 2 weeks, in some of the nightly loads, an error has ocurred in any of the following activities: Lookup, Copy activity, Web activity due to the same error:

"
Error code
2403

Details
Get access token from MSI failed for Datafactory ddc-df-prod, region we. Please verify resource url is valid and retry. Details: Acquire MI token from AAD failed. ErrorCode: request_throttled, Message: Your app has been throttled by AAD due to too many requests. To avoid this, cache your tokens see https://aka.ms/msal-net-throttling.
"

As the error message suggests, I should "cache my tokens". I am not really sure that I am able to achieve this in data factory.
My pipelines are failing "sometimes" and I have not changed anything in the last weeks.

Thanks in advanced,
Gilfredo

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,689 questions
{count} votes