How to identify status of an activity in Azure data factory

Question

How to identify status of an activity in Azure data factory

Priya Jha 901

Hi All,

I was following below architecture for removing duplicate activities to get the error message of every activity for logging purpose.

For logging error at every activity level i used the following code to concat error messages from every activity
@markus.bohland@hotmail.de (activity('Act_1').error?.message,'^|',activity('Act_2').error?.message,'^|',activity('Act_3').error?.message,'^|',activity('Act_4').error?.message)

But because we are using Skipped along with completion dependency, the upper concat expression fails with the following error when an activity is skipped:
The expression 'concat(activity('Act_2').error?.message)' cannot be evaluated because property 'error' cannot be selected.

Is there any way, i can get error messages for every activity in the above architecture?

Also is there any flag or way to identify whether an activity went through a skipped ,or completion case if all 2 flows are mapping to same output?

MartinJaffer-MSFT 26,161 Reputation points

2021-07-13T22:36:18.57+00:00
I almost forgot your other ask @Priya Jha .

When an activity is skipped, we can't exactly fetch its status. However, the absence of status is also a strong indicator.

This means I can combine the null-safe operator, and the coalesce function, to figure out the status of an activity.

@{coalesce(activity('Lookup1')?.Status,'Skipped')}

Is this close enough to your ask of determining whether skipped or completed?

Answer accepted by question author

0 additional answers

Your answer

MartinJaffer-MSFT 26,161 Reputation points

2021-07-13T22:36:18.57+00:00

I almost forgot your other ask @Priya Jha .

When an activity is skipped, we can't exactly fetch its status. However, the absence of status is also a strong indicator.

This means I can combine the null-safe operator, and the coalesce function, to figure out the status of an activity.

@{coalesce(activity('Lookup1')?.Status,'Skipped')}

Is this close enough to your ask of determining whether skipped or completed?

Answer 1

Nandan Hegde 36,716 MVP Volunteer Moderator

Hey,
Since you have used Skipped logic, there would not be any output because of which you are getting the error.
You can validate whether there is any output or not by using '?':
activity('Act_2')?.error?.message

My guess is there is no flag available based on my knowledge.

@MartinJaffer-MSFT @KranthiPakala-MSFT @HimanshuSinha @Nasreen Akter : please correct me if I am wrong or any other thoughts on the same.

MartinJaffer-MSFT 26,161 Reputation points

2021-07-13T20:34:04.557+00:00
So close, @Priya Jha , you almost had it. @Nandan Hegde is correct.

When the activity has not run, it cannot make an error. The ?. makes a thing null-safe.

You wrote

activity('Act_2').error?.message

Nandan wrote

activity('Act_2')?.error?.message

The difference is where the ?. is located. With the ?. before message it is anticipating whether message is null. However it cannot get to that point if error does not exist / is null. This is why placing the ?. before error fixes the issue. The ?. before message may not be necessary.

activity('Act_2')?.error.message

Thank you for the quick answer Nandan. :-)
Nandan Hegde 36,716 Reputation points MVP Volunteer Moderator

2021-07-14T09:40:14.33+00:00

Hey @MartinJaffer-MSFT ,
Based on the above Arch, my guess is there should be .? before message as well because lets consider the scenario wherein the first 2 activities are successful and the last one fails .
And since there is no error message for the initial 2 ones the concat statement might fail without .?

Share via

How to identify status of an activity in Azure data factory

0 additional answers

Your answer