the relation of coordinate between the color image and the depth image

Question

These day, I uese k4arecorder.exe to get the .mkv files. I extracted each frame of color image and depth image from the video. But now I have to accomplish the function that find a specific point in the color image, obtain its coordinates in the color image, and obtain the depth value at the corresponding position in the corresponding depth image. For example, I have get the coordinate of right hand in the rgb image. I want to know how to find the position of the right hand in the depth image and get its depth value based on this coordinate. Thanks!

Answer 1

Hi @Jianxin xiao Thank you for posting the question on this forum. I don't think there is any out of the box approaches provided by the Body tracking SDK to achieve what you are looking for. The closest out of the box approach I see is to use the body tracking index map which includes the instance segmentation map for each body in the depth camera capture. Each pixel maps to the corresponding pixel in the depth or IR image. The value for each pixel represents which body the pixel belongs to. It can be either background (value K4ABT_BODY_INDEX_MAP_BACKGROUND) or the index of a detected k4abt_body_t.

You may use the function k4abt_frame_get_body_index_map() to access the body index map. Below is a sample snippet on how to use it in the code.

k4a_image_t body_index_map = k4abt_frame_get_body_index_map(body_frame);
...  // Do your work with the body index map
k4a_image_release(body_index_map);

You can get more information on this functionality from the following resources.

• Access data in body frame
• Azure Kinect body tracking index map