Azure Data Factory (ADF) DRP - self hosted integration runtime question

Question

Is it possible to assume that the following DRP setup will work:

• ADF is created and metadata are available in backup region
• SHIR is created in a primary ADF with 2 VMs in primary region and 2 VMs in backup region (2 VMs in backup region will be switched off and turned on only in case of DRP)
• will SHIR metadata be available in backup region ?

As per Microsoft documentation (https://learn.microsoft.com/en-us/azure/architecture/example-scenario/analytics/pipelines-disaster-recovery) it is stated that:
Azure managed automatic failover doesn't apply to self-hosted integration runtime (SHIR), because the infrastructure is customer-managed. For guidance on setting up multiple nodes for higher availability with SHIR,

So I need a clarity if SHIR metadata will be available in backup region and in case of disaster it will work after switching on VMs from backup region?

Answer 1

Hi @Marcin Parzy
Thank you for your question.

You have outlined a valid DRP setup approach, and I’ll clarify how Self-hosted Integration Runtime (SHIR) behaves in such scenarios:

SHIR Metadata Availability:

Yes, SHIR metadata is stored in the cloud (in your ADF instance), not on the VMs themselves. So as long as your Azure Data Factory metadata is accessible in the backup region, the SHIR configuration (such as linked integration runtimes, authentication keys, etc.) will remain intact.

DRP with Paired Nodes:

In your setup, where SHIR is registered with both primary and backup region VMs under the same SHIR node group, this design is supported. The SHIR can operate as a single logical unit even if nodes span across regions - as long as they're registered using the same authentication key.

During a Failover:

When the VMs in the primary region go down, you can bring up the backup region VMs (already registered to the same SHIR). These nodes will automatically start processing workloads as they join the SHIR cluster, assuming they have connectivity to required services and data sources.

Important Considerations:

• Ensure the backup region VMs are pre-registered with the same SHIR key.
• Validate network configurations (firewall rules, DNS resolution, etc.) in both regions.
• Plan for potential latency or performance differences when operating across regions.

For more details, Microsoft’s official guidance is here: Pipelines disaster recovery – Azure Architecture Center

Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 2

Is there any way to test it (access metadata from backup region)?