Synapse can't access storage (403)

Question

Hello everyone,

I am facing a firewall issue that prevents my pipeline from working correctly :

The pipeline (Azure Synapse) is supposed to be triggered when files are deposited in a blob storage directory, copying them from 'ENTREE' to 'EN_COURS'.

Everything was working fine when the firewall was set to 'All Networks'

Here are the IP and Azure Instances allowed in the networking :

After setting up an ip whitelisting so only us and our clients could access the portal I am facing this error after the pipeline is triggered by a file being deposited in the blob directory (ENTREE) :

"ErrorCode=AzureBlobOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Blob operation Failed. ContainerName: https://dkdatalakecanon.blob.core.windows.net/dk-storage-canon-medical, path: ENTREE/PRODUITS/UL_Resah_2019_070_LOT20_A000_AV5.xlsx.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The remote server returned an error: (403) Forbidden.,Source=,''Type=Microsoft.WindowsAzure.Storage.StorageException,Message=The remote server returned an error: (403) Forbidden.,Source=Microsoft.WindowsAzure.Storage,StorageExtendedMessage=RequestId:2c9299fd-201e-001d-49fa-4f8d18000000 Time:Thu, 14 Apr 2022 12:21:54 GMT,,''Type=System.Net.WebException,Message=The remote server returned an error: (403) Forbidden.,Source=Microsoft.WindowsAzure.Storage,'", "failureType": "UserError", "target": "

After adding the IPs found here (https://learn.microsoft.com/en-us/azure/cosmos-db/how-to-configure-firewall), I have another error :

{ "errorCode": "2200", "message": "ErrorCode=AzureBlobOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Blob operation Failed. ContainerName: https://dkdatalakecanon.blob.core.windows.net/dk-storage-canon-medical, path: ENTREE/PRODUITS/UL_Resah_2019-070_LOT20_A000_AV5.xlsx.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The remote server returned an error: (403) Forbidden.,Source=,''Type=Microsoft.WindowsAzure.Storage.StorageException,Message=The remote server returned an error: (403) Forbidden.,Source=Microsoft.WindowsAzure.Storage,StorageExtendedMessage=RequestId:276787cf-601e-006e-48d2-50d58b000000 Time:Fri, 15 Apr 2022 14:06:57 GMT,,''Type=System.Net.WebException,Message=The remote server returned an error: (403) Forbidden.,Source=Microsoft.WindowsAzure.Storage,'", "failureType": "UserError", "target": "Copie_Entree_To_En_Cours", "details": [] }

Thanks in advance for your help !

Answer 1

Hello @PRADEEPCHEEKATLA ,

My IP address is whitelisted as I can access the sftp server that deposit the file into the blob.

Answer 2

Try to create a private endpoint to the storage account to access from Synapse work space - this will use the backbone rather than public net