Auto Update for Self Hosted Integration Runtime

Question

Hi,

May I know if I had an SHIR with 1 node and the auto update kick in, what will happen to my running pipelines? Will it crash the running pipelines or it will wait until the data factory go into an idle state? Also what will happen when it in the middle of the update and a pipeline trigger kick in? Normally how long will the downtime will last(I know this also depend on the network)? But I need to know on average how long it will last.

I tested the multiple nodes setup for the SHIR, but don't have a good experience with it. It required all the nodes having the same version installed. At some point, I even had a hard time to sync the nodes even having the same version installed.

Thanks.

Answer 1

Hi @Lee, Steven Hong Sing
Welcome to Microsoft Q&A platform and thanks for posting your question here.

Scenario

Imagine you have a Self-hosted Integration Runtime with a single node that is responsible for executing data movement activities in your Azure Data Factory. You have a pipeline Pipeline A that is currently running a data copy activity from an on-premises SQL Server to Azure Blob Storage.

Auto-Update Behavior

Now, if an auto-update is triggered for the SHIR while Pipeline A is running, here’s what will happen:

Running Pipelines

The SHIR will attempt to gracefully complete any ongoing activities. If it cannot finish before the update starts, the activities will be paused. The pipeline itself won’t crash; it will be in a suspended state until the SHIR is back online.

Idle State

The SHIR doesn’t necessarily wait for the ADF to be in an idle state. It prioritizes ongoing activities but will proceed with the update once it reaches a safe point to pause.

Pipeline Trigger During Update

If a new trigger for Pipeline_B is activated while the SHIR is updating, the trigger will be acknowledged, but the pipeline won’t start until the SHIR is available again.

Downtime Duration

The downtime for an auto-update typically lasts a few minutes, but this can vary. On average, you might expect it to last anywhere from 5 to 15 minutes, depending on the factors you mentioned like network speed and update size.

Multiple Nodes Setup

Regarding the multiple node's setup, it’s designed for high availability and load balancing. All nodes in the SHIR must have the same version to ensure consistency and proper load distribution. Here’s an example of how syncing issues might occur:

Example of Syncing Issues

You have a two-node SHIR setup: Node_1 and Node_2. Both nodes are supposed to auto-update to version X.1. Node_1 successfully updates to X.1, but Node_2 encounters an issue and remains on version X.0. As a result, Node_2 becomes out of sync, and you might experience issues with load balancing or failover capabilities.

Resolving Syncing Issues

To resolve such syncing issues, you can:

• Ensure that auto-update is enabled on all nodes.
• Manually intervene to update any node that didn’t auto-update correctly.
• Check for any network or permission issues that might prevent a node from updating.

The Self-hosted Integration Runtime is designed to handle updates without disrupting ongoing processes severely. Pipelines will pause rather than crash during updates, triggers will queue up and wait for the SHIR to become available, and downtime is generally short to minimize impact on your workflows.

Reference
https://learn.microsoft.com/en-us/azure/data-factory/self-hosted-integration-runtime-auto-update

Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.