Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,696 questions
Azure Synapse Analytics - Data exfiltration error when storage account and synapse workspace are in the same tenant and resource group
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(41.6, 31%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECL%3C/text%3E%3C/svg%3E)
Carrillo, Lino
1
Reputation point
We have a Synapse Workspace whose primary/default data lake storage account is private. Both the workspace and the storage account are in the same tenant and same resource group:
- Workspace has a managed private endpoint to the storage account
- Workspace has Blob Storage Data Contributor access to the storage account
- I have Blob Storage Data Contributor access to the storage account, am a Synapse Administrator, have Contributor access to the workspace, and am the SQL AD Administrator
- Data exfiltration protection is on
We created the workspace and the storage account using ARM template (stored alongside other resources as infra-as-code) and we're trying to read a dedicated SQL Pool table into a Spark pool using the Azure Synapse Apache Spark to Synapse SQL connector. When we first create it and then add the managed private endpoint to the storage account, we are able to read from the dedicated SQL Pool table into Spark pool without issues. However, when we reapply the ARM template to make changes to some other resource, we are unable to read from the dedicated SQL Pool table claiming:
com.microsoft.spark.sqlanalytics.SQLAnalyticsConnectorException: com.microsoft.sqlserver.jdbc.SQLServerException: Data exfiltration to '<storage account name>.dfs.core.windows.net' is blocked. Add destination to allowed list for data exfiltration and try again.
I do some checks to see if the managed private endpoint connectivity broke, but we all the checks are green on both Synapse Studio and the storage account private endpoints. We are also able to resolve the storage account dns name to an IP that I assume resides inside the managed VNet where the spark pool operates.
We are able to fix this issue by deleting the managed private endpoint and creating it again, but the moment we re-apply our infra-as-code we're back to a broken integration. Is this a bug or are we missing something?
{count} votes
-
Carrillo, Lino 1 Reputation point
2021-12-04T00:35:33.003+00:00 I can confirm this is only an issue when I redeploy the "Microsoft.Synapse/workspaces" resource. I've removed its innards and modified it to be declared as an "existing resource" and the issue is no longer happening. All other sub-resources are still being tracked and applied via infra-as-code.
This is even stranger since I checked using "az deployment sub what-if ..." and it was showing no changes for the synapse workspace resource. However, I would re-deploy the infra-as-code (with "no synapse workspace resource changes") and it would cause the issue.
What's going on?
To be even more clear, we are using "az deployment sub create ..." with Azure Pipelines, Azure CLI Task, and Azure Resource Manager service connection to deploy our infra-as-code.
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 26%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHM%3C/text%3E%3C/svg%3E)
HimanshuSinha-msft 19,386 Reputation points Microsoft Employee2021-12-08T20:30:09.163+00:00 Hello @Carrillo, Lino ,
Thanks for the patience , unfortunately we are not getting the kind of response from the team here . if you have a support plan you may file a support ticket, else could you please send an email to azcommunity@microsoft.com with the below details, so that we can create a one-time-free support ticket for you to work closely on this matter.
Subscription ID:
Subject : Attn Himanshu
Please let me know once you have done the same.Thanks
Himanshu -
Carrillo, Lino 1 Reputation point
2021-12-09T21:47:51.43+00:00 We have opened a support ticket. I'll report back here once I get more information.
Thanks!
-
Carrillo, Lino 1 Reputation point
2022-04-07T00:23:38.39+00:00 Back in February of this year we got confirmation from Azure Support that there was this issue was due to a bug in Azure . It has been fixed and I've confirmed in my environments that the managed private endpoint connectivity doesn't break anymore.
Sign in to comment