Rate limit exceeded Error - When creating a pipeline run from LA using azuredatafactory connection

Question

I have a "ApiConnection" action in Logic App (using azuredatafactory connector), where I create a pipeline run in a data factory. However, frequently, I see this error message :

{
"message": "Rate limit is exceeded, clientRequestId: xxxx"
}

1. Is this error occurring because the ADF connector is not able to handle the load, or is it that the data factory is not able to handle the load ?
2. do suggest recommended actions to take in this scenario.

Thanks,
Kothai.

Answer 1

@Kothai Ramanathan Thanks for using Microsoft Q&A !!
You could be getting this error as each connector in logic app has its own API call limits which would result in following errors. For ADF connector, throttling limits are defined as 100 calls per minute per connection. Refer to the Throttling Limits for ADF connector.

Common suggestions are:

• Use separate connection for each logic app even if the underlying resource is same.
• Even within same logic app, use separate connection for different type of actions on the same resource.
• Instead calling action in for-each loop, possibly call child Logic Apps and implement action call there.
• While fetching large amount of data, use Pagination
• If Pagination is not available, create a logic to pull data in appropriate batches.
• Implement appropriate delays in-between.
• Despite all care taken, implement proper error handling and retry logic to handle 429 event
  Handle errors and exceptions in Azure Logic Apps

Also, if usage requirement is really high or mission critical , you can go for ISE – fixed cost model where most of the limits are removed or uplifted. As it is an isolated environment, user can predict and control the load.

Please let me know if you have any other questions.