I created an Azure IR within Synapse in order that I could set up a TTL of 5 minutes to be associated with the triggered pipeline. The pipeline has a simple data flow where the source and sink both use the same linked service. And that linked service is associated with the new IR with TTL. I trigger the pipeline but the IR monitor only shows it processing in the default AutoResolveIntegrationRuntime. Why would this pipeline not run in the new IR with TTL?

Hi @Tim , Welcome to Microsoft Q&A platform and thankyou for posting your query. As I understand your question , you have created Azure IR with TTL option and used the same in the linked service of your dataflow. However, in the monitor tab , it shows pipeline is using the default AutoResolveIntegrationRuntime which is not the expected result. I have raised your concern with our internal product team, will keep you posted upon hearing back from them . Thanks for your patience!

Triggered pipeline not using the expected Integration Runtime with TTL

Accepted answer

AnnuKumari-MSFT 30,676 Reputation points Microsoft Employee

2022-04-25T06:45:11.343+00:00

Hi @Tim ,
Welcome to Microsoft Q&A platform and thankyou for posting your query.

As I understand your question , you have created Azure IR with TTL option and used the same in the linked service of your dataflow. However, in the monitor tab , it shows pipeline is using the default AutoResolveIntegrationRuntime which is not the expected result.

I have raised your concern with our internal product team, will keep you posted upon hearing back from them . Thanks for your patience!
Please sign in to rate this answer.
Tim 31 Reputation points

2022-04-25T12:03:12.977+00:00

Hello, I just discovered that I had not set the IR property in the pipeline itself. I had thought the IR path was solely determined by the Linked Services involved in the pipeline. After making this change the IR with TTL is now being used as expected.

AnnuKumari-MSFT 30,676 Reputation points Microsoft Employee

2022-04-25T12:18:08.647+00:00

Hi @Tim ,
Glad to know you were able to find the resolution. Thanks for sharing the same. However, just confirming the workaround for anyone who finds similar issue . While calling the dataflow in the pipeline , in the data flow settings, we need to select the IR to be used to execute the data flow. Here is the screenshot of the same:

Please do consider clicking Accept Answer as accepted answers help community as well.

Tim 31 Reputation points

2022-04-25T12:26:18.71+00:00

That is correct of course. My only other question is regarding the linked servers that are used in the pipeline activities... That is, must the linked servers also be associated with the new IR (integrationRuntime2 in your image) in order for the pipeline to use integrationRuntime2?

AnnuKumari-MSFT 30,676 Reputation points Microsoft Employee

2022-04-25T12:42:45.3+00:00

Hi @Tim ,
It's not necessary to have the linked service point to the IR with TTL in order to leverage the feature of TTL re-utilizing the existing cluster while executing the data flow . We just need to select the IR with TTL option in the pipeline level. Hope this helps!
Sign in to comment

Triggered pipeline not using the expected Integration Runtime with TTL

0 additional answers