Synapse - Switch activity takes longer than its sub processes

Question

Synapse - Switch activity takes longer than its sub processes

Hi team, I have a switch activity that contains few conditions and one of the flow contains two child activities as shown below.
As seen below, the individual child activity completes, put together in 1 minute while the overall switch activity is taking 4+m to finish. IR used by two child activities looks fine with the logs and handles back the successful message to switch in shown time. What could be the reason for this? This pattern is intermittent though, but leads to inconsistent run time.

User's image

Thanks, Padmanabhan

QuantumCache 20,346 Reputation points

2024-01-24T18:33:39.4333333+00:00

@Padmanabhan Thirugnanasambandam If the switch activity is processing large volumes of data, it can cause delays in the activity. What is the amount of data are we looking at ? If this is intermittent issue, can we check for any changes happened to the Data handling or the pipeline?
Padmanabhan Thirugnanasambandam 5 Reputation points

2024-01-24T23:00:46.7133333+00:00

Hi SatishBoddu-MSFT, Thanks for the prompt response, the Switch is one of the activity under a For each loop that runs in parallel for 40 iterations. In this particular instance, the row count of two child activities are under 100 rows and completed in under 1 min. I am still wonder what the switch was doing for rest 3+ mins when the data load is already done.

I can also confirm there was no change made to the pipelines. Today, I am not seeing this issue yet. Thanks, Padmanabhan
Padmanabhan Thirugnanasambandam 5 Reputation points

2024-01-29T01:21:53.2466667+00:00
QuantumCache 20,346 Reputation points

2024-01-29T17:47:21.88+00:00

@Padmanabhan Thirugnanasambandam Could you please re-attach the image ? Thanks
Oh, ok, i got the image! looking into this!
QuantumCache 20,346 Reputation points

2024-01-30T00:45:00.3066667+00:00

Hello @@Padmanabhan Thirugnanasambandam Did you try this scenario from ADF? instead of synapse? What is the region of your Synapse? I remember that the Switch activity may sometimes take few extra seconds , but few minutes is not able to reproduce!!! Out of curiosity question: What is the blocker on your side due to switch Activity taking longer time? is this blocking any business flow? or are you just benchmarking any activity?
QuantumCache 20,346 Reputation points

2024-02-01T02:29:28.24+00:00

Hello @Padmanabhan Thirugnanasambandam Just checking if we are still connected on this discussion? Please let us know if you need to add more info so that we better assist you!
Padmanabhan Thirugnanasambandam 5 Reputation points

2024-02-02T04:48:57.4566667+00:00

Hi SatishBoddu-MSFT

I haven't yet checked this with ADF. we are with Australia East. Below one has taken 5+mins today with Switch. Is there a log that I can access (not the Self hosted compute being used here) to understand what could be happening with Switch?
QuantumCache 20,346 Reputation points

2024-02-02T05:14:13.4466667+00:00
Hello @Padmanabhan Thirugnanasambandam Thanks for your patience in this matter!

Please send us an email, so that we can provide one-time free technical support to your subscription to troubleshoot the issue in deep.

Please send an email with the below details, so that we can work offline on this matter.

Email Subject : Attn Satish Boddu

Email Body content must include the below:

Thread URL: Link to this thread.

Your Azure Subscription ID: <This is must to be provided to us>
Padmanabhan Thirugnanasambandam 5 Reputation points

2024-02-05T05:17:29.47+00:00

Thanks SatishBoddu-MSFT for your support.

2 answers

Your answer

QuantumCache 20,346 Reputation points

2024-01-24T18:33:39.4333333+00:00

@Padmanabhan Thirugnanasambandam If the switch activity is processing large volumes of data, it can cause delays in the activity. What is the amount of data are we looking at ? If this is intermittent issue, can we check for any changes happened to the Data handling or the pipeline?
Padmanabhan Thirugnanasambandam 5 Reputation points

2024-01-24T23:00:46.7133333+00:00

Hi SatishBoddu-MSFT, Thanks for the prompt response, the Switch is one of the activity under a For each loop that runs in parallel for 40 iterations. In this particular instance, the row count of two child activities are under 100 rows and completed in under 1 min. I am still wonder what the switch was doing for rest 3+ mins when the data load is already done.

I can also confirm there was no change made to the pipelines. Today, I am not seeing this issue yet. Thanks, Padmanabhan
Padmanabhan Thirugnanasambandam 5 Reputation points

2024-01-29T01:21:53.2466667+00:00
QuantumCache 20,346 Reputation points

2024-01-29T17:47:21.88+00:00

@Padmanabhan Thirugnanasambandam Could you please re-attach the image ? Thanks
Oh, ok, i got the image! looking into this!
QuantumCache 20,346 Reputation points

2024-01-30T00:45:00.3066667+00:00

Hello @@Padmanabhan Thirugnanasambandam Did you try this scenario from ADF? instead of synapse? What is the region of your Synapse? I remember that the Switch activity may sometimes take few extra seconds , but few minutes is not able to reproduce!!! Out of curiosity question: What is the blocker on your side due to switch Activity taking longer time? is this blocking any business flow? or are you just benchmarking any activity?
QuantumCache 20,346 Reputation points

2024-02-01T02:29:28.24+00:00

Hello @Padmanabhan Thirugnanasambandam Just checking if we are still connected on this discussion? Please let us know if you need to add more info so that we better assist you!
Padmanabhan Thirugnanasambandam 5 Reputation points

2024-02-02T04:48:57.4566667+00:00

Hi SatishBoddu-MSFT

I haven't yet checked this with ADF. we are with Australia East. Below one has taken 5+mins today with Switch. Is there a log that I can access (not the Self hosted compute being used here) to understand what could be happening with Switch?
QuantumCache 20,346 Reputation points

2024-02-02T05:14:13.4466667+00:00

Hello @Padmanabhan Thirugnanasambandam Thanks for your patience in this matter!

Please send us an email, so that we can provide one-time free technical support to your subscription to troubleshoot the issue in deep.

Please send an email with the below details, so that we can work offline on this matter.

Email Subject : Attn Satish Boddu

Email Body content must include the below:

Thread URL: Link to this thread.

Your Azure Subscription ID: <This is must to be provided to us>
Padmanabhan Thirugnanasambandam 5 Reputation points

2024-02-05T05:17:29.47+00:00

Thanks SatishBoddu-MSFT for your support.

Answer 1

QuantumCache 20,346

Hi Padmanabhan Thirugnanasambandam Thanks for more context! Could you please add screenshot of your pipeline Activities , so that i can try the repro? Let me quickly test some samples on my side! Keep us posted if you see further variations! Have done some testing and i see the Switch Activity is mostly consistent! May be we have to dig deeper into your scenario!!!
User's image

Answer 2

Padmanabhan Thirugnanasambandam 5

Hi SatishBoddu-MSFT, Thanks for the prompt response. I could still see the issue and below is another pattern of data load and the Switch take longer than the child processess. I am using data load activity under the switch statement.

User's image

Two different patterns of data load and same issue with timings

Padmanabhan Thirugnanasambandam 5 Reputation points

2024-02-20T01:19:49.45+00:00

SatishBoddu-MSFT
Hi Satish,
FYI, Here is the service ticket raised via my company mail: 2402200030000140
QuantumCache 20,346 Reputation points

2024-02-21T03:26:48.7133333+00:00

Padmanabhan Thirugnanasambandam Thanks for the response, the technical team will continue to help you on the case!

Share via

Synapse - Switch activity takes longer than its sub processes

2 answers

Your answer