How to automatice in AzureDevops the creation of a synapse managed private endpoint across environments?

Question

Hi, I have created a synapse managed private endpoint in synapse studio, and I don't find the proper documentation to create this by AzureDevops CICD pipeline for non-development environment.

The only docu I have found is regarding the ps script New-AzSynapseManagedPrivateEndpoint, but it requires a definition file which is not described in the docu.

Answer 1

Hi @PATRICIA FUENTES ,

Thanks for using Microsoft Q&A!!
If my understanding is correct you want to create a managed private endpoint using PowerShell cmdlet which you want to use in the Azure DevOps automation.
I agree with you that the documentation doesn't talk about the definition file. I will provide your feedback internally to the CLI team to provide clarity on the documentation, however, you can use the definition json file in the below format to create a managed private endpoint.

{  
    "name": "AzureBlobStoragePE",  
    "properties": {  
        "privateLinkResourceId": "/subscriptions/{SubscriptionId}/resourceGroups/{ResourceGroupName}/providers/Microsoft.Storage/storageAccounts/{StorageAccountName}",  
        "groupId": "blob"  
    }  
}  

Cmdlet Output using the above json file

Above json will create private endpoint against a storage account. You need to change the privateLinkResourceId and groupId based on the resource type you are using.
For example -
If you want to create for Datalake Storage then you need to use change groupId to dfs and add fqdns property with the corresponding server details.

Also, groupId is the type of sub-resource that your private endpoint will be able to access e.g. blob, dfs, sql etc.
You could look into documentation to get some idea on the different properties which you could pass in your json file.

Also, you can check the json of existing managed private endpoints in your Synapse Workspace to set specific values for your definition file or use PowerShell cmdlet Get-AzSynapseManagedPrivateEndpoint to get details. (See screenshot below)


Other References - Private Link Resources - List By Storage Account

Please let me know if you have any other questions.

Thanks
Saurabh

----------

Please do not forget to "Accept the answer" wherever the information provided helps you to help others in the community.

Answer 2

Hi,

I am getting this error from ADF when I am copying a long table to parquet file, but not always. Any suggestion? Thanks a lot.

Failure happened on 'Source' side. ErrorCode=UserErrorFailedFileOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Upload file failed at path -XXXXX.parquet.,Source=Microsoft.DataTransfer.Common,''Type=System.Data.SqlClient.SqlException,Message=A transport-level error has occurred when receiving results from the server. (provider: TCP Provider, error: 0 - The specified network name is no longer available.),Source=.Net SqlClient Data Provider,SqlErrorNumber=64,Class=20,ErrorCode=-2146232060,State=0,Errors=[{Class=20,Number=64,State=0,Message=A transport-level error has occurred when receiving results from the server. (provider: TCP Provider, error: 0 - The specified network name is no longer available.),},],''Type=System.ComponentModel.Win32Exception,Message=The specified network name is no longer available,Source=,'