max concurrent connections in adf and synapse

Question

Hi friends, what is max concurrent connections in adf and synapse in copy activity and when it is compulsory to have a value?

Answer 1

@Anshal - Thanks for the question and using MS Q&A paltform.

In Azure Data Factory (ADF), the maximum number of concurrent connections for a copy activity depends on the type of source and sink data stores you are using. The maximum number of concurrent connections is determined by the data store's capabilities and limitations. For example, if you are copying data from an on-premises SQL Server database to an Azure SQL Database, the maximum number of concurrent connections is determined by the SQL Server's maximum number of concurrent connections and the Azure SQL Database's maximum number of concurrent connections.

In Azure Synapse Analytics, the maximum number of concurrent connections for a copy activity is determined by the number of concurrent queries you have configured for your service. By default, the maximum number of concurrent queries is set to 32, but you can increase this limit up to a maximum of 128 concurrent queries.

It is not always compulsory to specify a value for the maximum number of concurrent connections in ADF or Synapse. In most cases, the default values will work fine. However, if you are experiencing performance issues or if you need to optimize your data transfer, you may need to adjust the maximum number of concurrent connections.

For example, if you are copying data from a large on-premises database to an Azure data store, you may need to increase the maximum number of concurrent connections to improve performance. Similarly, if you are copying data from multiple sources to a single sink, you may need to adjust the maximum number of concurrent connections to ensure that the copy activity does not exceed the data store's maximum number of concurrent connections.

In general, it's a good practice to monitor the number of concurrent connections and activities in your ADF and Synapse environments to ensure that you are not exceeding any limits and to optimize performance.

If you read the Microsoft documentation it says the following: The maximum number of concurrent runs the pipeline can have. By default, there is no maximum. If the concurrency limit is reached, additional pipeline runs are queued until earlier ones complete.

And you may checkout this article to How to prevent concurrent pipeline execution?

Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.