Cannot delete blobs from ADLS gen2 when connected via Private Endpoint

spencer 31 Reputation points
2021-10-26T14:51:29.69+00:00

Hello,

A colleague and I noticed that although we can list, view, and update blobs in an Azure Data Lake Storage Gen2 account, we cannot delete them when connected via a private endpoint and the default network access rule is set to "Selected networks"/Deny.

All actions work as expected when default network access rule is set to allow / "All networks".

Steps to reproduce:

  1. Create ADLS Gen2 account
  2. Create Private Endpoint connection, associated with a VNet, DNS, etc.
  3. Set the default network access rule for the ADLS Gen2 to deny.
  4. Create blobs in the account
  5. Attempt to delete blobs within the account

Error from Azure Storage Explorer / azcopy:

`{
"message": "\"failed to perform remove command due to error: cannot start job due to error: cannot verify resource due to error: -> github.com/Azure/azure-storage- > azcopy/v10/azbfs.newStorageError, /home/vsts/work/1/s/azbfs/zc_storage_error.go:41\n===== RESPONSE ERROR (ServiceCode=AuthorizationFailure) =====\nDescription=403 This request is not authorized to perform this operation., Details: (none)\n HEAD https://xxxxxxxxxxxxxxxxxxxxxxxxx.dfs.core.windows.net/mynewtestcontainer/query_data.csv?timeout=901\\n Authorization: REDACTED\n User-Agent: [Microsoft Azure Storage Explorer, 1.20.1, win32, azcopy-node, 2.0.0, win32, AzCopy/10.11.0 Azure-Storage/0.1 (go1.15; Windows_NT)]\n X-Ms-Client-Request-Id: [80fa79ef-8c51-4450-608e-62dbbf4eb47a]\n X-Ms-Date: [Tue, 26 Oct 2021 14:46:20 GMT]\n X-Ms-Version: [2018-11-09]\n --------------------------------------------------------------------------------\n RESPONSE Status: 403 This request is not authorized to perform this operation.\n Date: [Tue, 26 Oct 2021 14:46:19 GMT]\n Server: [Windows-Azure-HDFS/1.0 Microsoft-HTTPAPI/2.0]\n X-Ms-Error-Code: [AuthorizationFailure]\n X-Ms-Request-Id: [7402d82d-301f-0030-1378-ca5e31000000]\n X-Ms-Version: [2018-11-09]\n\n\n.\n\""

}`

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,351 questions
Azure Private Link
Azure Private Link
An Azure service that provides private connectivity from a virtual network to Azure platform as a service, customer-owned, or Microsoft partner services.
469 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Siva Villa 280 Reputation points Microsoft Employee
    2023-07-13T08:03:48.56+00:00

    If you create a private endpoint for the Data Lake Storage Gen2 storage resource, then you should also create one for the Blob Storage resource. By creating a private endpoint for both resources (Blob & DFS), you ensure that all operations can complete successfully.
    https://learn.microsoft.com/en-us/azure/storage/common/storage-private-endpoints#creating-a-private-endpoint

    3 people found this answer helpful.