What is the best way to detect in real time reflective targets on the Hololens 2 ?

Question

What is the best way to detect in real time reflective targets on the Hololens 2 ?

Nicolas Lorrain 0

Hello,

I am currently working on a project which requires to detect reflective targets on objects, and create holograms based on these targets.

I know I need Research Mode to get access to IR cameras, and I use this github https://github.com/microsoft/HoloLens2ForCV to help. I can get infrared camera pixels and then our image processing algorithms could do the trick.

I don't want to use Unity or any other engine, so I will be using the C++/WinRT (no OpenXR as I need to access Research Mode, only possible with WinRT)

Is this the best way to do this or is there another appropriate way that you could think of ?

Moreover, do you have any code that could help me in my work ? https://github.com/microsoft/HoloLens2ForCV help me greatly to access sensors but I don't seem to find some code that would help me create an hologram fixed in the world (I only found some in OpenXR and I have difficulties understanding the documentation about spatial anchors and coordinate systems - I understand how it works but I don't get how to change the position of an object independently of the device)

Thank you !

Nicolas Lorrain 0 Reputation points

2023-03-16T17:40:05.0766667+00:00

Another question, it appears that there that the IR sensor gives some noise inside reflective targets (see the images), is it possible to get an image without this default ? 1 (1).PNG

Nicolas Lorrain 0 Reputation points

2023-03-16T17:40:05.0766667+00:00

Another question, it appears that there that the IR sensor gives some noise inside reflective targets (see the images), is it possible to get an image without this default ? 1 (1).PNG

Answer 1

Zuocheng Wang - MSFT 1,251 Microsoft Vendor

Hello, Welcome to Microsoft Q&A,

Yes, accessing the Depth & IR Camera via research mode on HoloLens is in the right direction. For using WinRT to implement Spatial Anchor, you can refer to Coordinate systems in DirectX - Mixed Reality | Microsoft Learn.

For noise in the IR sensor, have both versions DEPTH_AHAT and DEPTH_LONG_THROW been tried?

If the response is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Nicolas Lorrain 0 Reputation points

2023-03-20T17:33:25.5666667+00:00

Hello, thank you for the answer.

DEPTH_AHAT does not detect reflective versions as good the DEPTH_LONG_THROW does, and I must detect the targets even when I'm more than 1 meter away from them. See the images for comparison

Depth Long Throw :

L

AHAT :

And do you know if a project on Github or somewhere exists with WinRT which creates an object which keeps static in space when moving the Hololens (I want an object that does not follow the device, like a real object would) ? I am having trouble using Spatial anchors even with the documentation

Thank you !
Nicolas Lorrain 0 Reputation points

2023-03-20T20:17:49.2533333+00:00

To clarify my problem, what I don't understand is how to create a spatial anchor at a fixed point in the world. I know I can use SpatialAnchor::TryCreateRelativeTo but I need to give a coordinate system, and the only one I can get is the one of the device thanks to SpatialLocator::GetDefault().
What am I missing here ?
Zuocheng Wang - MSFT 1,251 Reputation points Microsoft Vendor

2023-03-22T07:20:24.98+00:00

Regarding the coordinate system needed to create a Spatial Anchor, you can get it through SpatialStationaryFrameOfReference or SpatialLocatorAttachedFrameOfReference, which one you use depends on your needs.

There is currently no official GitHub sample for implementing Local Spatial Anchor using C++/WinRT, but you can refer to the creation process of Local Anchor in this Azure Spatial Anchor sample - Create & locate anchors in C++/WinRT - Azure Spatial Anchors | Microsoft Learn. The m_attachedReferenceFrame used here is obtained through spatialLocator.CreateAttachedFrameOfReferenceAtCurrentHeading(). This is repo - azure-spatial-anchors-samples/HoloLens/DirectX at master · Azure/azure-spatial-anchors-samples · GitHub.

Regarding the issue of noise in IR sensors, you can open a ticket via https://aka.ms/hlsupport for further discussion.

What is the best way to detect in real time reflective targets on the Hololens 2 ?

1 answer

Activity