Express route and Azure Data Lake gen 2

Christophe Laudou 96 Reputation points


I am doing some testing around accessing Azure Data Lake Gen 2 from an On Premise hadhoop
I need to access the service via Express Route
What is the recommended setup if anyone has been through a configuration?


Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
993 questions
Azure ExpressRoute
Azure ExpressRoute
An Azure service that provides private connections between Azure datacenters and infrastructure, either on premises or in a colocation environment.
211 questions
{count} votes

Accepted answer
  1. Christophe Laudou 96 Reputation points

    Using Private End Point on a VNet / Subnet seems to be the right solution in order to connect simple from an On Premise setup of WANDisco to the Azure Data Lake Service Gen 2 storage account.

    the key element is to modify the hosts file or DNS entry in order to use the Private IP to resolve the ADLS hostname


    Implementation steps can be found here

1 additional answer

Sort by: Most helpful
  1. PRADEEPCHEEKATLA-MSFT 59,331 Reputation points Microsoft Employee

    @Christophe Laudou Welcome to the Microsoft Q&A platform.

    These are different sources of data and the different ways in which that data can be ingested into a Data Lake Storage Gen2 account.


    Data stored in on-premises or IaaS Hadoop clusters:

    Large amounts of data may be stored in existing Hadoop clusters, locally on machines using HDFS. The Hadoop clusters may be in an on-premises deployment or may be within an IaaS cluster on Azure. There could be requirements to copy such data to Azure Data Lake Storage Gen2 for a one-off approach or in a recurring fashion. There are various options that you can use to achieve this. Below is a list of alternatives and the associated trade-offs.


    Really large datasets:

    For uploading datasets that range in several terabytes, using the methods described above can sometimes be slow and costly. In such cases, you can use Azure ExpressRoute.

    Azure ExpressRoute lets you create private connections between Azure data centers and infrastructure on your premises. This provides a reliable option for transferring large amounts of data. To learn more, see Azure ExpressRoute documentation.

    Reference: Migrate data from on-premise Hadoop to Azure Storage and Using Azure Data Lake Storage Gen2 for big data requirements

    Hope this helps. Do let us know if you any further queries.


    Do click on "Accept Answer" and Upvote on the post that helps you, this can be beneficial to other community members.