How do i get frame rate or other performance metrics on when running networks on the azure percept devkit?

bm21_416 21

I have been running networks on Azure percept devkit and i am interested in knowing how to measure performance such as FPS for the networks. I have not found any documentation of that anywhere, will be a big help if i can get some info on this.

QuantumCache 20,186 Reputation points

2021-06-25T23:24:44.097+00:00

Hello @bm21_416 Welcome to Microsoft Q&A Platform and thanks for your query. Community SME's on this topic or our team will review your scenario and circle back at the possible earliest time.
bm21_416 21 Reputation points

2021-06-29T20:36:17.217+00:00

Hello, still looking for some help on this please
QuantumCache 20,186 Reputation points

2021-06-30T01:50:18.927+00:00

My apologies, our team is looking into this.
Mike McCoy - MSFT 606 Reputation points

2021-07-01T22:45:57.233+00:00

Hi @bm21_416 ,

You can access your model's telemetry via these instructions:

https://learn.microsoft.com/en-us/azure/azure-percept/how-to-view-telemetry

Let me know if this gives you the data and information you're looking for.

Thanks!
bm21_416 21 Reputation points

2021-07-07T16:33:35.993+00:00

Hey so, i tried the Telemetry as suggested (thank you for pointing me in that direction). It provides the results of the model which i can already see by querying the device over SSH like this "watch -n 0.1 docker logs --tail 100 azureeyemodule".

What i am looking for is if there is a way to measure frames per second accurately for the azure percept devkit?
bm21_416 21 Reputation points

2021-07-07T17:02:14.987+00:00

Is there a way for me to print logs myself? It seems by default it does not produce the FPS values i am looking for.
Mike McCoy - MSFT 606 Reputation points

2021-07-07T23:53:53.807+00:00

Hi @bm21_416 ,

I'm checking with someone who has more expertise in this area. Stay tuned...

-Mike

Accepted answer

Mike McCoy - MSFT 606 Reputation points

2021-07-12T18:23:25.657+00:00

@bm21_416 - unfortunately, the Azure Percept DK does not currently have an API in the Myriad X SDK suitable for profiling neural networks.

However, you could try to calculate the FPS by taking the difference of two consecutive frames in the log output. For example, the below two frames are about 400 milliseconds apart. Note that seqno is the frame ID in consecutive order. So if subsequent frame IDs are not one integer apart, it means a frame was dropped.

nn: seqno=2, ts=1615811179194168848, []
nn: seqno=3, ts=1615811179235832767, []

Thanks,

-Mike
Please sign in to rate this answer.

1 person found this answer helpful.
bm21_416 21 Reputation points

2021-07-13T16:40:10.303+00:00

Thanks I will check this out. By the way what batch sizes are these networks set to? in the model URLs here: https://github.com/microsoft/azure-percept-advanced-development

bm21_416 21 Reputation points

2021-07-13T17:15:27.087+00:00

A bit confused on your math here what unit is the ts value in?

QuantumCache 20,186 Reputation points

2021-07-17T06:46:42.09+00:00

Tagging @Mike McCoy - MSFT for quick attention!

Mike McCoy - MSFT 606 Reputation points

2021-07-19T16:32:11.427+00:00

Hi @bm21_416

The Timestamps are Unix timestamps in nanoseconds since January 1st 1970. So when you see ts=1615920044588919099, what that means is Tue, Mar 16, 2021 @ 18:40:44 GMT. Note that we use 64-bit unsigned integers for the timestamp, so in case the device is still functioning 17 years from now, we won't overflow.

I realized I made a mistake when calculating the example I previously gave you. The formula for this example is (later frame - earlier frame)/1000000 = the time difference in milliseconds. So my previous example should be (1615811179235832767 - 1615811179194168848)/1000000 = ~42ms.

Here is a handy Unix timestamp converter: https://www.unixtimestamp.com/

bm21_416 21 Reputation points

2021-07-19T18:00:57.9+00:00

Thank you!

Mike McCoy - MSFT 606 Reputation points

2021-07-19T18:03:48.057+00:00

No problem! Please "accept" this Answer :-)

Thanks!
Sign in to comment

Share via

How do i get frame rate or other performance metrics on when running networks on the azure percept devkit?

0 additional answers