question

colinrippeyfinarne avatar image
0 Votes"
colinrippeyfinarne asked HimanshuSinha-MSFT commented

Azure Synapse Managed VNET and connectivity to the serverless endpoint issues from a pipeline

I have a scenario that (I'm sure) a lot of folk will want to use but I am running into issues inside a Synapse workspace that uses a Managed VNET.

If we can take a simple Synapse workspace that is NOT on a Managed VNET first.

I create a Linked Service inside synapse that I configure as an Azure SQL, I use the current serverless endpoint as the fqdn and connect via Managed Identity. At this point click Test Connection it works.

I then create an integration dataset that "points" to the new linked service, the master database.

I then create an integration dataset of type Parquet and point to a new folder on the connected data lake.

I then create a Pipeline and add the Copy Data, for the source I use the integration dataset and use a simple query say

SELECT * from sys.databases

I set the sink as the Parquet dataset.

This is a very simple "can I execute a query against the current serverless endpoint and get back results as part of a pipeline".


We have been using this connection quite a lot, we have a ton of data in our lake we expose using the serverless endpoint to external consumers such as Power BI etc. and we have also written some pipelines that consume the serverless endpoint as if it was an Azure SQL database.

Every part of the connectivity/debug experience all works from Test Connection to Preview Data. If I run the pipeline I get a Parquet file created so all good.

If I try this in a Synapse workspace that uses a Managed VNET I get no issues with Test Connection, Preview Data, but when I try to run the pipeline I see the pipeline run and attempts to connect to the SQL linked service but never succeeds (basically the pipeline sits for minutes trying to connect to the serverless endpoint and I then cancel it).


So I am missing something in terms of how the integration runtimes connect to the serverless endpoints on a Managed VNET it seems.

azure-synapse-analytics
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @colinrippeyfinarne ,
Thanks for the ask and using the Microsoft Q&A platform .
At this time, we are reaching out to the internal team to get some help on this . We will update you once we hear back from them.
Thanks
Himanshu

1 Vote 1 ·

Hello @colinrippeyfinarne ,
Thanks for the patience , unfortunately we are not getting the kind of response from the team here . if you have a support plan you may file a support ticket, else could you please send an email to azcommunity@microsoft.com with the below details, so that we can create a one-time-free support ticket for you to work closely on this matter.
Subscription ID:
Subject : Attn Himanshu
Please let me know once you have done the same.

Thanks

0 Votes 0 ·

0 Answers