ADF api delay

Question

how to fix the ADF api latency where parallel apis sending to microsoft crm .. how to fix this latency api is taking time to send the data

Answer 1

@Vineet S

Thanks for reaching out to Microsoft Q&A.

Dealing with latency issues in Azure Data Factory (ADF) when sending data to Microsoft CRM can be challenging. Here are some steps you can take to address this:

1. Optimize API Calls: Ensure that your API calls are optimized. This includes checking the performance of the external REST APIs, as they might be causing delays
2. Parallelism and Concurrency: Increase the degree of parallelism and concurrency in your ADF pipeline settings. This can help in processing multiple API calls simultaneously, reducing overall latency
3. Timeout Settings: Review and adjust the timeout settings for your activities. Sometimes, increasing the timeout can help manage longer-running processes more effectively
4. Batch Processing: If possible, batch your data before sending it to the CRM. This can reduce the number of API calls and improve performance
5. Monitoring and Logging: Enable detailed logging and monitoring to identify where the delays are occurring. This can help you pinpoint specific bottlenecks in your pipeline
6. Resource Allocation: Ensure that your ADF instance has sufficient resources allocated. Sometimes, increasing the Data Integration Units (DIUs) can help improve performance

Is there a specific part of your pipeline where you're noticing the most delay?