Share via

Create On Demand HDInsight from Data Factory in an existing VNet

Beniamin Murza 26 Reputation points
2020-09-23T07:07:55.617+00:00

The Azure Data Factory can create an on-demand HDInsight (linked service), which is automatically deleted after the job running on it finishes.
Is it possible to have this on-demand HDInsight created in an existing VNet?

Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
207 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,199 questions
0 comments
Accepted answer
  1. HarithaMaddi-MSFT 10,136 Reputation points
    2020-09-23T13:18:54.667+00:00

    Hi @Beniamin Murza ,

    Welcome to Microsoft Q&A Platform. Thanks for posting the query.

    As per my understanding, this would work only by installing a self-hosted integration runtime (SHIR) in the same virtual network. This can be achieved by creating a new VM joining to VNET and install SHIR on it. This IR can be used while configuring HDInsight Linked Service which will ensure that cluster is created in VNET. Ref: tutorial-transform-data-hive-virtual-network-portal

    I am working with Product team to understand any other possible alternatives to achieve this and will get back to you once I get more details from them.

    Hope this helps! Please let us know if this does not align with the requirement or for further queries and we will be glad to assist.

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest