Azure Integration Runtime Error on Auto Resolve or East Europe region

Question

Hi all.

Last month I started to receive this error on 2 of my Azure Integration Runtime ( Azure 1 and Azure 2).
"There are substantial concurrent MappingDataflow executions which is causing failures due to throttling under Integration Runtime 'AzureCluster1-Quick', ActivityId: '5dadd0b2-03ba-47fb-8177-21e2ea576615'"

Here is some of my investigations:

• All 4 of my runtimes had Auto Resolve region.
• There is no concurent executions, got failure on the very first step or Debug via default IR.
• If I switch the same Data Flows to my other Azure IR (3 or 4) works better but sometimes have same error.
• If I change region to East Europe - same error.
• If I change region to other regions - no error.
• My Azure Database placed East Europe.

Currently got error every time. Have the same infrastructure on Dev/Staging/Preprod/Prod environments. There are problems only with Staging one. Previously had some migration between resourse groups for this Data Factory (changing subscriptions). Can be the reason. If so - please provide me instructions how to avoid it on other env's.

Here are some other Activity Id's:
Azure Runtime 1
'5dadd0b2-03ba-47fb-8177-21e2ea576615'
'455c1e7d-89a1-4752-b5db-a9282e3e56b5'
'007e8d92-5b41-455f-8a43-6ad05568e73b'

Azure Runtime 2
'337ce6dd-0b50-4c5a-a3fd-cdb970342db4'
'f7f44127-1b00-4009-84ea-30c65cb178c3'
'ec85b394-1388-4dde-98f9-6134acc6e688'

Will be thanful for any help.

Answer 1

Hi @Illia Klievtsov ,

Welcome to Microsoft Q&A Platform. Thank you for posting query on Microsoft Q&A Platform.

You may have reached integration runtimes capacity limit or you might be running large amount of data flow using same integration runtime at same time.

You should consider running your pipeline data flow at different times or split your data flow execution's to across multiple integration runtimes.

Kindly check below thread where detailed resolution is discussed.
https://learn.microsoft.com/en-us/answers/questions/282303/azure-datafactory-error.html

Hope this will help. Please let us know if any further queries. Thank you.

------------------------------

• Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.
• Want a reminder to come back and check responses? Here is how to subscribe to a notification.