Using Private Endpoints in Conjuction with a Virtual Network Gateway

Michael Mallok 21 Reputation points
2022-08-01T13:46:31.447+00:00

Hello Community,

First of all, I already found a thread concerning this problem but it was two years old and the proposed solution didn't work for me.

So here is the situation:
I want to access an Azure Data Lake Storage Gen2 exclusevely through private endpoints, no public connections allowed.
I provisioned a VNet for the purpose of accommodating the private endpoints from the storage account.
This already works well with Data Factory and from VMs sitting in the connected VNet (for testing purposes).

The problem arises when trying to connect from a local machine.
I'm using a virtual network gateway to connect to the VNet (using certificates) from a Windows 10 machine which mostly works but I can't access blobs in the storage account.
When trying to access through the portal, I get a 403 (This request is not authorized to perform this operation.) and I get similar errors when using Power BI Desktop.

I think that I found the reason for this problem:
The local machine is connected to the VNet but (by design) in a separate IP space which doesn't have access to the private endpoints.
Is there any way to bridge this gap? Or is there another (cheap and relatively easy) way to connect to my ADLS through private endpoints? Did I miss something?

The solution a user proposed was creating the gateway in one vnet and the private endpoints in a second vnet and connecting them via peering. I tried that but it didn't change anything.

Cheers,
Michael

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,338 questions
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,684 questions
Azure VPN Gateway
Azure VPN Gateway
An Azure service that enables the connection of on-premises networks to Azure through site-to-site virtual private networks.
1,378 questions
Accepted answer
  1. SaiKishor-MSFT 17,181 Reputation points
    2022-08-10T19:42:28.3+00:00

    @Michael Mallok I apologize for the delay in responding to your post. Appreciate your response to my questions.

    Please note that the the Storage account needs to have access from the Vnet if you want to be able to access it from your on-premises. This is because the private endpoint is assigned an IP address from the IP address range of the Vnet.

    To answer your questions-

    • This aforementioned assigned IP address is not part of the VNet (10.5.0.0/16) or any of its Subnets. Thus, there is no built-in way of accessing the Private Endpoints. Is there a way to bridge that gap without setting up a VM that acts as DNS Server?

    Unfortunately, you only have two options but both need some configuration on your custom DNS server i.e., You should configure your DNS server to delegate your private link subdomain to the private DNS zone for the VNet, or configure the A records for StorageAccountA.privatelink.blob.core.windows.net with the private endpoint IP address.

    Please refer to these articles for more information on configuring a private endpoint for Azure Storage:

    1. https://www.youtube.com/watch?v=V8PjtCTTT6c
    2. This article refers to configuring DNS when using private endpoints: https://learn.microsoft.com/en-us/azure/private-link/private-endpoint-dns
    3. https://learn.microsoft.com/en-us/azure/private-link/private-endpoint-dns#on-premises-workloads-using-a-dns-forwarder
      • The requests coming from my local machine try to use the public IPs of the Storage Account instead of the private ones provided by the Private Links of the Private Endpoints. Can that be the case? And if so, is that something that has to be fixed on the local machine or in the routing inside of the Azure VNet?

    This is because the DNS server that your local machines use does not have an A record for the privatelink DNS name of the storage account which should look something like:

    StorageAccountA.privatelink.blob.core.windows.net A 10.1.1.5

    At that point, the traffic will try to go to the Vnet(since the private endpoint IP will belong to the Vnet address range) via the VPN/Express Route and it will eventually be able to hit the Storage account privately. Right now, it is using the Public DNS which resolves to the Public IP.

    Please fix the 2 issues which are, allowing the Vnet into the Storage Account as well as resolving the DNS to the private IP. Hope this helps.

    Let us know if you have any further questions and we will be glad to assist you further. Thank you!

    Remember:

    Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.

    Want a reminder to come back and check responses? Here is how to subscribe to a notification.

    2 people found this answer helpful.

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Michael Mallok 21 Reputation points
    2022-08-16T06:56:48.313+00:00

    Thank you very much for your detailed answer! There is obviously more to it than I originally thought. Thanks for the links and pointers in the right direction!

    0 comments