Hello, I am currently working on a project which requires to detect reflective targets on objects, and create holograms based on these targets. I know I need Research Mode to get access to IR cameras, and I use this github https://github.com/microsoft/HoloLens2ForCV to help. I can get infrared camera pixels and then our image processing algorithms could do the trick. I don't want to use Unity or any other engine, so I will be using the C++/WinRT (no OpenXR as I need to access Research Mode, only possible with WinRT) Is this the best way to do this or is there another appropriate way that you could think of ? Moreover, do you have any code that could help me in my work ? https://github.com/microsoft/HoloLens2ForCV help me greatly to access sensors but I don't seem to find some code that would help me create an hologram fixed in the world (I only found some in OpenXR and I have difficulties understanding the documentation about spatial anchors and coordinate systems - I understand how it works but I don't get how to change the position of an object independently of the device) Thank you !

Hello, Welcome to Microsoft Q&A, Yes, accessing the Depth & IR Camera via research mode on HoloLens is in the right direction. For using WinRT to implement Spatial Anchor, you can refer to Coordinate systems in DirectX - Mixed Reality | Microsoft Learn . For noise in the IR sensor, have both versions DEPTH_AHAT and DEPTH_LONG_THROW been tried? If the response is helpful, please click " Accept Answer " and upvote it. Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

What is the best way to detect in real time reflective targets on the Hololens 2 ?

Accepted answer

Zuocheng Wang - MSFT 3,091 Reputation points Microsoft Vendor

2023-03-17T09:38:18.81+00:00

Hello, Welcome to Microsoft Q&A,

Yes, accessing the Depth & IR Camera via research mode on HoloLens is in the right direction. For using WinRT to implement Spatial Anchor, you can refer to Coordinate systems in DirectX - Mixed Reality | Microsoft Learn.

For noise in the IR sensor, have both versions DEPTH_AHAT and DEPTH_LONG_THROW been tried?

If the response is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.
Please sign in to rate this answer.

1 person found this answer helpful.
Nicolas Lorrain 20 Reputation points

2023-03-20T17:33:25.5666667+00:00

Hello, thank you for the answer.

DEPTH_AHAT does not detect reflective versions as good the DEPTH_LONG_THROW does, and I must detect the targets even when I'm more than 1 meter away from them. See the images for comparison

Depth Long Throw :

L

AHAT :

And do you know if a project on Github or somewhere exists with WinRT which creates an object which keeps static in space when moving the Hololens (I want an object that does not follow the device, like a real object would) ? I am having trouble using Spatial anchors even with the documentation

Thank you !

Nicolas Lorrain 20 Reputation points

2023-03-20T20:17:49.2533333+00:00

To clarify my problem, what I don't understand is how to create a spatial anchor at a fixed point in the world. I know I can use SpatialAnchor::TryCreateRelativeTo but I need to give a coordinate system, and the only one I can get is the one of the device thanks to SpatialLocator::GetDefault().
What am I missing here ?

Zuocheng Wang - MSFT 3,091 Reputation points Microsoft Vendor

2023-03-22T07:20:24.98+00:00

Regarding the coordinate system needed to create a Spatial Anchor, you can get it through SpatialStationaryFrameOfReference or SpatialLocatorAttachedFrameOfReference, which one you use depends on your needs.

There is currently no official GitHub sample for implementing Local Spatial Anchor using C++/WinRT, but you can refer to the creation process of Local Anchor in this Azure Spatial Anchor sample - Create & locate anchors in C++/WinRT - Azure Spatial Anchors | Microsoft Learn. The m_attachedReferenceFrame used here is obtained through spatialLocator.CreateAttachedFrameOfReferenceAtCurrentHeading(). This is repo - azure-spatial-anchors-samples/HoloLens/DirectX at master · Azure/azure-spatial-anchors-samples · GitHub.

Regarding the issue of noise in IR sensors, you can open a ticket via https://aka.ms/hlsupport for further discussion.

Nicolas Lorrain 20 Reputation points

2023-03-23T14:13:06.33+00:00

Thank you for your comments, they were helpful. I decided to go without anchors for now but your explanations helped me understand better the use of it.

For the noise in IR sensors, I solved it ! For anyone wondering, in https://github.com/microsoft/HoloLens2ForCV, they use a mask of value 0x80 in the ConvertDepthPixel function for depth long throw sensor (in SlateCameraRenderer.cpp) which is responsible for this behavior
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

What is the best way to detect in real time reflective targets on the Hololens 2 ?

1 additional answer

Your answer