Azure Synapse Notebook request to Cognitive services translator

Question

I've got a notebook in Azure Synapse that I use to translate data using Azure Cognitive Services. The code used to work, but I had to rebuild by Synapse workspace and it no longer works. It looks like the Spark environment has no access to the internet or other Azure services, as I just get a connection timeout. I don't know how to configure access, as I configured Synapse with a managed VNet.

import os, requests, uuid, json
subscription_key = "XXXXXXXXXXXXXXXXXXXXXXXXXX"
constructed_url = "https://api-eur.cognitive.microsofttranslator.com/translate?api- 
  version=3.0&from=en&to=af"
headers = {
  'Ocp-Apim-Subscription-Key': subscription_key,
  'Ocp-Apim-Subscription-Region' : 'northeurope',
  'Content-type': 'application/json',
  'X-ClientTraceId': str(uuid.uuid4())
}
body = [{
  'text' : 'hello'
}]
request = requests.post(constructed_url, headers=headers, json=body)
response = request.json()
print(json.dumps(response, sort_keys=True, indent=4, separators=(',', ': ')))

Result:

ConnectTimeout: HTTPSConnectionPool(host='api-eur.cognitive.microsofttranslator.com', port=443): Max retries exceeded with url: /translate?api-version=3.0&from=en&to=af (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f83aa63fc18>, 'Connection to api-eur.cognitive.microsofttranslator.com timed out. (connect timeout=10)'))

I have configure a Managed Private Endpoint from Synapse to Cognitive Services, but I don't know how to access that from the notebook, as the address is not visible anywhere. Any ideas?
If anyone has a better way of translating data by perhaps using Data Flow transformations, I'd also be keen to use that.

Answer 1

@Pierre-Andre van Leeuwen

I believe your new workspace is using data exfiltration protection with managed virtual network where former one was not using the same and thus you are getting this error while using translator API.

Those links below describe the 2 flavors of Synapse VNETs - first one is with VNET only

1. Managed virtual network - Azure Synapse Analytics
2. Data exfiltration protection for Azure Synapse Analytics workspaces - Azure Synapse Analytics
   
3. Add data exfiltration protection when creating your workspace

Products team is still working to officially support Cognitive Services with exfiltration protection and providing public documentation, but the changes to create a managed PE for Cognitive Services is already available. Just note that not all Cognitive Services support private endpoints, so it is important to make sure the translator API supports private endpoints for that scenario. Also, looks like microsofttranslator.com would be a public IP.

The short official answer at this time is that it is not yet supported in exfiltration protected workspaces.

So, you need to try using your code on synapse workspace with vnet only (without exfilteration). Hope this helps.

----------

Please do not forget to "Accept the answer" wherever the information provided helps you to help others in the community.