Hi, I just need to know what the 'time' referes in azure data factory output logs ? it is not equal with the actual time.

@Mustafa Ispir I have determined the "time" property found under each stage, is how many milliseconds each stage took to process. I ran a data flow, and extracted the activity output, and then checked under monitoring details as seen in below screenshots. For ease of reading, the activity output is abbreviated.

output log time

Mustafa Ispir 41

Hi,

I just need to know what the 'time' referes in azure data factory output logs ?

User's image

it is not equal with the actual time.

User's image

MartinJaffer-MSFT 26,021 Reputation points

2023-02-08T05:07:50.6633333+00:00

@Mustafa Ispir Hello and welcome to Microsoft Q&A.

As I understand you want a definition of this "time" metric. Is this coming from the output of Data Flow activity in pipeline, or something else?
MartinJaffer-MSFT 26,021 Reputation points

2023-02-17T18:05:14.5233333+00:00

Did my answer help you? If it solved your issue, please mark as accepted answer. If not let me know how I may better assist.

1 answer

MartinJaffer-MSFT 26,021 Reputation points

2023-02-08T05:44:55.5566667+00:00

@Mustafa Ispir

I have determined the "time" property found under each stage, is how many milliseconds each stage took to process.

I ran a data flow, and extracted the activity output, and then checked under monitoring details as seen in below screenshots. For ease of reading, the activity output is abbreviated.
Please sign in to rate this answer.
Mustafa Ispir 41 Reputation points

2023-02-08T09:24:54.78+00:00

Hi Martin,

Thanks for the answer.

But as i see,processung time is 12 second and sum of logs is 6+1+1=8.

How can i get this 12 second ?

Mustafa Ispir 41 Reputation points

2023-02-08T09:34:28.01+00:00

And another question is about time difference.When i debug a pipeline i see runtime process is like below 4.42

But when i click it's log i see like 28 second.

What they referred ?

Mustafa Ispir 41 Reputation points

2023-02-13T11:20:34.9633333+00:00

Does no one knows it really ?

MartinJaffer-MSFT 26,021 Reputation points

2023-02-17T20:04:12.9633333+00:00

@Mustafa Ispir Processing time is only one component of total time. There is time taken to allocate the compute resources and prepare the environment for actual work.

There is other overhead. There is write and read time. You can click the binoculars for some of the breakdown. Other breakdown can be found in the output.
Sign in to comment