Hi Please help I have been trying to understand , How to Log the Failure Message / Success Message in ADF in the CSV (Blob Storage) Getting Error The expression 'activity('Data flow1').output.rowsRead' cannot be evaluated because property 'rowsRead' doesn't exist, available properties are 'runStatus, effectiveIntegrationRuntime, billingReference, reportLineageToPurview'.

Hi @Kunal Bhatia , Thankyou for using Microsoft Q&A platform and posting your question. You need to use this expression : @activity('Data Flow1').Error.message Hope this will help. Please let us know if any further queries. ------------------------------ Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how Want a reminder to come back and check responses? Here is how to subscribe to a notification If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Logging Error in ADF - Microsoft Q&A

Accepted answer

AnnuKumari-MSFT 33,721 Reputation points Microsoft Employee

2022-01-31T10:46:07.66+00:00
Hi @Kunal Bhatia ,
Thankyou for using Microsoft Q&A platform and posting your question.

You need to use this expression : @activity('Data Flow1').Error.message

Hope this will help. Please let us know if any further queries.

------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you.
Original posters help the community find answers faster by identifying the correct answer. Here is how

Want a reminder to come back and check responses? Here is how to subscribe to a notification

If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
Please sign in to rate this answer.
Kunal Bhatia 71 Reputation points

2022-01-31T17:49:48.74+00:00

Thanks for the same ,
But Still I am not succeeded to save my logs in csv file in Blob.
I want to save the error message for troubleshoot later.

Also what would be the way to save the Success Logs also in file , Since I have been trying to resolve this from long time, But not succeeded yet.
Please suggest.

AnnuKumari-MSFT 33,721 Reputation points Microsoft Employee

2022-01-31T19:40:17.677+00:00

Hi @Kunal Bhatia ,

Create a dummy csv file with one column

Drag a copy activity. In source, select the dataset pointing to the dummy file.

Add additional column option in source and provide value as @variables('errormessage')

Create new dataset to store the errorMessage and select that in Sink

Go to mapping tab and import schema . Provide a string for the variable(can be any string). Delete the additional column from dummy dataset and run the pipeline

Please refer to the attached gif for more details:

If this answers your query, do click Accept Answer and Up-Vote for the same. Also, kindly take the satisfaction survey. And, if you have any further query do let us know

AnnuKumari-MSFT 33,721 Reputation points Microsoft Employee

2022-02-01T12:10:17.23+00:00

Hi @Kunal Bhatia ,
Just checking in to see if the above answer helped. If this answers your query, do click Accept Answer and Up-Vote for the same as accepted answer helps community as well. And, if you have any further query do let us know.

Kunal Bhatia 71 Reputation points

2022-02-01T15:46:27.347+00:00

Thank you very much for the support, it works.

Just wanted to confirm , the same way we can also Log the output on success. Like rowsread, rowscopied .. etc ??
we have to create the extra column for all that , right ? and pass in the variables query.

AnnuKumari-MSFT 33,721 Reputation points Microsoft Employee

2022-02-02T13:38:33.657+00:00

Hi @Kunal Bhatia ,
Glad to hear that it worked. Yes you can use same approach to log success details.
Your pipeline structure will look something like this :

Check the JSON output file and using online JSON formatter get the expression for the metrices you want to capture.
For instance, Expression for RowsWritten can be something like : @{activity('Data Flow1').output.runStatus.metrics.sink1.rowsWritten}
Similarly, for RowsRead : @{activity('Data Flow1').output.runStatus.metrics.sink1.sources.source1.rowsRead}
You need to create the expression taking a look at your JSON output .

Similar to the error logging process, add additional columns on dummy source dataset and in sink provide output dataset for successlog ( to capture header as well , check the First row as header option in Sink dataset).

If this answers your query, please do click Accept Answer and Up-Vote for the same as accepted answers help community as well. If you have any further query do let us know.

AnnuKumari-MSFT 33,721 Reputation points Microsoft Employee

2022-02-03T08:54:14.237+00:00

Hi @Kunal Bhatia ,
Thankyou for taking out your time to share your feedback. As I can see you have taken the survey and provided rating to us but the answer has still not been accepted. Looking forward to see the answer acceptance as accepted answer helps the community as well. Thanks!
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Logging Error in ADF

0 additional answers

Your answer