Synapse Managed Virtual Network Integration Runtime

Jeremy D. Busselman 36 Reputation points
2022-01-27T19:47:33.387+00:00

Utilizing a Managed Virtual Network helps from a security aspect, and I am aware that the runtime needs to spin up.

What I'm struggling to understand or find documentation on is that, for EACH copy or dataflow step, needs to spin up the runtime, so if I have 3 copy steps in one pipeline, and it takes 2 minutes to spin up a runtime, then the overall duration of the pipeline has 6 minutes added, JUST to wait in queue to spin up. Imagine taking that by numerous pipelines.

Isn't there a way to spin up a runtime at the beginning of a batch cycle for all pipelines to use, so it's 2 minutes extra for the batch cycle, vs an exponential number based on spinning up for each and every step that needs it?

I can't imagine why anyone would choose this solution if you have to suffer the pain of this happening. There has to be something I'm missing??

Thanks

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,447 questions
{count} votes