Internal Server Error related to number of activities

Question

Internal Server Error related to number of activities

Will Trickett 0

Hello - I've recently taken advantage of the 80 activity limit to add logging to a pipeline in Synapse. When I try to debug the pipeline, I get this error:

User's image

The pipeline contains one lookup activity from a control table, and then a switch activity that runs a number of different activities in each switch based on the return value from the lookup.

The pipeline validates, and when I run the switch statements in their own separate pipelines they succeed.

When debugging, I tried deleting each switch and adding the switches and activities one-by-one. The issue appears to be related to the number of activities. No matter which activities I run, this error returns when I have 63 total pipeline activities. 62 activities work, and then the additional 1 causes the error.

What could be the cause of this? There is no error message, so very difficult to diagnose.

1 answer

Your answer

Answer 1

Smaran Thoomu 22,115 Microsoft External Staff

Hi @Will Trickett

Thanks for the question and using MS Q&A platform.

Based on the information you provided, it seems that you are encountering an internal server error related to the number of activities in your Synapse pipeline. You mentioned that the error occurs when you have 63 total pipeline activities, but 62 activities work without any issues.

One possible cause of this issue could be related to the resource limits of your Synapse workspace. Synapse workspaces have resource limits that can be exceeded if you have too many activities in your pipeline. You can check the resource limits of your Synapse workspace by navigating to the "Overview" tab of your Synapse workspace in the Azure portal.

Another possible cause of this issue could be related to the memory limits of your Synapse workspace. If you have too many activities in your pipeline, it could cause your workspace to run out of memory, resulting in an internal server error. You can check the memory limits of your Synapse workspace by navigating to the "Monitoring" tab of your Synapse workspace in the Azure portal.

To resolve this issue, you can try optimizing your pipeline by reducing the number of activities or splitting your pipeline into smaller pipelines. You can also try increasing the resource limits or memory limits of your Synapse workspace if necessary.

Hope this helps. Do let us know if you any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Will Trickett 0 Reputation points

2024-04-12T14:52:27.3866667+00:00

Hi @Smaran Thoomu

Thank you for the quick reply.

I navigated to the "Overview" tab but don't see any details around workspace resource limits. Could you point me to where exactly that's found?

This doesn't appear to be related to anything such as vCore capacity (we have 1000 available and very few used) or other compute capacity. These jobs run successfully when created as child pipelines - just not altogether in one pipeline.

I also don't see any details on the "Monitoring" tab about memory limits. We have never had an out-of-memory error before when running jobs, and I'm not sure where that detail is found?

Is it possible this is a bug since Microsoft recently moved to allowing up to 80 activities in one pipeline? Or an upper limit that's not specified? Looking at the announcement below, it appears we can define up to 80 activities, but not actually run that many. Is there an upper limit of how many can be run sequentially?

https://techcommunity.microsoft.com/t5/azure-data-factory-blog/data-factory-increases-maximum-activities-per-pipeline-to-80/ba-p/4096418

We'd like to take advantage of that new upper limit to simplify our pipelines, but at this point it appears we would need to keep everything as child pipelines.
Smaran Thoomu 22,115 Reputation points Microsoft External Staff

2024-04-15T05:03:01.6366667+00:00

Will Trickett Thank you for the information. As below in the overview you can see your workspace resources. currently, I don't have any.

However, you can also check the resource limits and memory limits of your Synapse workspace by contacting Azure support.

Regarding your question, it is possible that there is an upper limit on the number of activities that can be run sequentially. This limit is not specified in the documentation, and it is possible that it is related to the resource limits or memory limits of your Synapse workspace.

To further investigate this issue, I would recommend that you open a support ticket with Azure support. They will be able to provide you with more information about the resource limits and memory limits of your Synapse workspace, as well as help you troubleshoot the issue with the number of activities in your pipeline.

I hope this helps! Let me know if you have any further questions.

Share via

Internal Server Error related to number of activities

1 answer

Your answer