Synapse Integration Runtime Configuration

Nagesh CL 696 Reputation points
2023-01-20T08:53:13.21+00:00

Hello Team,

Is there any guidelines about configuration of the integration runtimes based on the data volumes? Currently we are dealing with ingesting data from SAP with few tables having data upto 80 Million Rows. Hence seeking advice or guidance on configuration of the integration runtime.

Thanks.

Regards,

Nagesh CL

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,375 questions
{count} votes

Accepted answer
  1. AnnuKumari-MSFT 34,556 Reputation points Microsoft Employee Moderator
    2023-01-23T06:47:54.39+00:00

    Hi Nagesh CL ,

    Thankyou for using Microsoft Q&A platform and thanks for posting your question here.

    As I understand your query, you want to know about the best practices for configuration of Self hosted Integration runtime when you are dealing with large volume of data. Please let me know if that is not the ask here.

    You can use multiple nodes in self hosted Integration Runtime to distribute the load between multiple machines when dealing with large volume of data.

    You can associate a self-hosted integration runtime with multiple on-premises machines or virtual machines in Azure. These machines are called nodes. You can have up to four nodes associated with a self-hosted integration runtime. The benefits of having multiple nodes on on-premises machines that have a gateway installed for a logical gateway are:

    • Higher availability of the self-hosted integration runtime so that it's no longer the single point of failure in your big data solution or cloud data integration. This availability helps ensure continuity when you use up to four nodes.
    • Improved performance and throughput during data movement between on-premises and cloud data stores. Get more information on performance comparisons.

    You can associate multiple nodes by installing the self-hosted integration runtime software from Download Center. Then, register it by using either of the authentication keys that were obtained from the New-AzDataFactoryV2IntegrationRuntimeKey cmdlet, as described in the tutorial.

    Note:

    • You don't need to create a new self-hosted integration runtime to associate each node. You can install the self-hosted integration runtime on another machine and register it by using the same authentication key.
    • Before you add another node for high availability and scalability, ensure that the Remote access to intranet option is enabled on the first node. To do so, select Microsoft Integration Runtime Configuration Manager > Settings > Remote access to intranet.

    For more information , check the official documentation on High availability and scalability of SHIR.

    Watch out this video that talks about how to install SHIR in VM with multiple nodes enablement.


    Hope it helps. Please do consider clicking Accept Answer and Up-Vote for the same as accepted answers help community as well.

    0 comments No comments

0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.