What is the best way to detect in real time reflective targets on the Hololens 2 ?

Nicolas Lorrain 20 Reputation points


I am currently working on a project which requires to detect reflective targets on objects, and create holograms based on these targets.

I know I need Research Mode to get access to IR cameras, and I use this github https://github.com/microsoft/HoloLens2ForCV to help. I can get infrared camera pixels and then our image processing algorithms could do the trick.

I don't want to use Unity or any other engine, so I will be using the C++/WinRT (no OpenXR as I need to access Research Mode, only possible with WinRT)

Is this the best way to do this or is there another appropriate way that you could think of ?

Moreover, do you have any code that could help me in my work ? https://github.com/microsoft/HoloLens2ForCV help me greatly to access sensors but I don't seem to find some code that would help me create an hologram fixed in the world (I only found some in OpenXR and I have difficulties understanding the documentation about spatial anchors and coordinate systems - I understand how it works but I don't get how to change the position of an object independently of the device)

Thank you !

HoloLens Development
HoloLens Development
HoloLens: A family of Microsoft self-contained, holographic devices that enable engagement with digital content and interaction with holograms in the surrounding environment.Development: The process of researching, productizing, and refining new or existing technologies.
380 questions
{count} votes

Accepted answer
  1. Zuocheng Wang - MSFT 3,081 Reputation points Microsoft Vendor

    Hello, Welcome to Microsoft Q&A,

    Yes, accessing the Depth & IR Camera via research mode on HoloLens is in the right direction. For using WinRT to implement Spatial Anchor, you can refer to Coordinate systems in DirectX - Mixed Reality | Microsoft Learn.

    For noise in the IR sensor, have both versions DEPTH_AHAT and DEPTH_LONG_THROW been tried?

    If the response is helpful, please click "Accept Answer" and upvote it.

    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

    1 person found this answer helpful.

1 additional answer

Sort by: Most helpful
  1. Deleted

    This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

    Comments have been turned off. Learn more