Capture an Error in a For Each loop in ADF

Question

Capture an Error in a For Each loop in ADF

Gareth Parry 20

Hi,

I have a pipeline that includes a ForEach loop. On occasion the pipeline fails in the for each loop, but the only error message i get is "Activity failed because an inner activity failed". How do i get to, or capture the actual error?

thanks

Gareth

Accepted answer

1 additional answer

Your answer

Answer 1

Venkat Reddy Navari 1,855 Microsoft External Staff Moderator

Hi @Gareth Parry When a ForEach loop fails in Azure Data Factory, you often get a generic message like "Activity failed because an inner activity failed." To capture the actual error details, here’s what you can do:

Check Inner Activities: Go to the Monitor tab, click on the ForEach activity, then expand each failed iteration. You'll be able to see which inner activity failed and why.
Use Expressions to Capture Errors: You can access detailed error info from the activity object. Normally you use:
```
   @activity('MyActivityName').output
```
But to get the error message when something fails, use:
```
   @activity('MyActivityName').error.message
```
This is especially useful inside error-handling logic, such as a Try-Catch pattern or an If Condition block, where you want to log or route errors.
Optional Logging: Consider appending the error message to a variable or logging it to a table or blob for future analysis.

Hope this helps. Do let us know if you any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Gareth Parry 20 Reputation points

2025-04-23T10:52:29.2666667+00:00

thanks for this. I have found the issue: a stored procedure times out. the timeout is set to 2 minutes but if i try to change it it tells me the maximum is 7 days, and the minimum is 10 minutes. How is it currently set to 2 minutes? i wanted to change to 2 m inutes 30
Venkat Reddy Navari 1,855 Reputation points Microsoft External Staff Moderator

2025-04-24T09:48:24.9733333+00:00
@Gareth Parry You're seeing a discrepancy because Azure Data Factory’s Stored Procedure activity has two timeout mechanisms at play:

ADF Activity Timeout (UI-controlled) This is what you set in the pipeline under "Timeout" (under Settings). The minimum value here is indeed 10 minutes, and the UI enforces that.

SQL Command Timeout (hidden, defaults) However, internally, ADF uses a default command timeout when executing stored procedures through the linked service. This is typically 30 seconds to 2 minutes, depending on the default setting of the ADO.NET or ODBC provider and your SQL environment.

So, in your case — even though the ADF activity timeout isn't set to 2 minutes — the command is likely timing out after 120 seconds (2 minutes) due to the default command timeout at the SQL client level, not ADF itself.

To explicitly control this:

Open your SQL linked service in ADF.

Add or update the connection string with:
Command Timeout=150
(150 seconds = 2 minutes 30 seconds) Save and republish the pipeline.

This will override the default command timeout and allow your stored procedure to run for 2.5 minutes before failing.

Hope this helps. Do let us know if you any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Gareth Parry 20 Reputation points

2025-04-24T15:03:18.51+00:00

thank you. i found this in the end and changed the linked service timeout from 30 seconds to 45. I haven't had any timeouts so far so hopefully this has resolved the issues
Venkat Reddy Navari 1,855 Reputation points Microsoft External Staff Moderator

2025-04-24T16:09:51.5133333+00:00

@Gareth Parry You're very welcome — glad to hear you tracked it down and updated the command timeout in the linked service from 30 to 45 seconds

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 2

Nandan Hegde 35,191 MVP Moderator

You need to capture the error message of all activities individually within the foreach and preserve it in a variable and you can reference it outside the for each. you can refer below blog : https://datasharkx.wordpress.com/2021/08/19/error-logging-and-the-art-of-avoiding-redundant-activities-in-azure-data-factory/ here to understand how to collate

Gareth Parry 20 Reputation points

2025-04-24T15:08:24.05+00:00

i thanks for this i think i have sorted out some logging, i have added a variable for all the individual activities in my foreach loop and connected them to the on fail part of the activity, collecting the error message. i have then added a variable for the on fail part of the for each loop itself and concatenated the 4 variables. I cant test it as i believe i have solved the issue, but it will be useful going forward if any other issues arise.

Share via

Capture an Error in a For Each loop in ADF

1 additional answer

Your answer