Azure Data Factory pipeline copy data to blob storage fails with ErrorCode=UserErrorFailedToCreateAzureBlobContainer

Question

I have Blob storage that is linked using managed private endpoint from ADF, and using Managed Vnet Integration runtime that points to managed private endpoint to blob storage.

from linked services, I'm using system assigned identity and I have givne 'Storage Blob Data Contributor' role to this target storage account.
Test Connection is successful.

but when I run very simple copy data pipeline (source Azure SQL database) sink dataset that uses linked service to blob storage container. it fails with below error message (storage name masked with xxxxxx)

Operation on target Test New SFTP Generate Extract failed: ErrorCode=UserErrorFailedToCreateAzureBlobContainer,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Unable to create Azure Blob container. Endpoint: https://xxxxxx.blob.core.windows.net/, Container Name: test.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.WindowsAzure.Storage.StorageException,Message=The remote name could not be resolved: 'xxxxxx.blob.core.windows.net',Source=Microsoft.WindowsAzure.Storage,''Type=System.Net.WebException,Message=The remote name could not be resolved: 'xxxxxx.blob.core.windows.net',Source=System,'

what would be the cause of this issue. using managed private endpoint, managed vnet integration runtime, TEST CONNECTION is SUCCESSFUL but pipeline fails

Note: I have created the exactly the same setup configuration in UAT Subscription it works fine???

Answer 1

Hi @madforchili , as you have mentioned the the source data connection works well, do try couple of steps delete the linked service as well as dataset from the sink

side and recreate, and more importantly did you check the blob storage firewall which ADF is trying to access and it (firewall) might blocking it. Thanks

Answer 2

Hi @madforchili ,

Thankyou for your question on Microsoft Q&A platform .

Please check your blob storage “Firewalls and virtual networks”. If Selected networks is checked, please try checking All networks. If you want to enable firewall then please make sure to whitelist the IP's that need to access your storage account and let us know if it helps.

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you.
  Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Answer 3

I am also facing same error it is working fine in one environment and failing in other environment

we already have containers created in the blob