Share via

No hand tracking in Unity holographic remoting (Hololens 2)

Xenotech Srl 1 Reputation point
2021-11-19T14:26:51.583+00:00

Hello everyone,

We're struggling with testing and debugging HL2 apps in Unity. Up until now testing's been a chore: change the code, build the VS project with Unity, open the project in VS, build and deploy the app to the device via USB. Each time we make just the slightest change it takes forever to test (building and deploying to device from VS takes several minutes!).
So we tried the holographic remoting tool in Unity, and it kinda works, but there's no way hand tracking is working, making it useless for us.

The setup is Unity 2020.3 LTS, Microsoft Mixed Reality OpenXR plugin, Unity OpenXR plugin, XR plugin management and XR Interaction Toolkit.
No MRTK.

We tried a ton of different configurations, changing Unity version, changing SDK version, changing plugins version, etc. Nothing worked so far.

Can someone please shed some light on this?
Thanks!

EDIT:
We've obviously gone through all the steps from Microsoft Learn (https://learn.microsoft.com/en-us/windows/mixed-reality/develop/unity/xr-project-setup?tabs=openxr)

EDIT2:
The Unity console throws an error: "Could not create a device for 'Microsoft HoloLens Hand OpenXR (XRInputV1)'(...)"

HoloLens Development
HoloLens Development
HoloLens: A family of Microsoft self-contained, holographic devices that enable engagement with digital content and interaction with holograms in the surrounding environment.Development: The process of researching, productizing, and refining new or existing technologies.
387 questions
0 comments

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Hernando Ren 2,166 Reputation points
    2021-11-22T08:02:34.05+00:00

    Hello, Welcome to Microsoft Q&A,

    It is recommended to leverage MRTK which has been mentioned in your post to enable the hand tracking feature in Unity. And the Hand tracking has been shipped as a basic building block in MRTK, one can short-circuit the developing process and concentrate instead on his business need. Also, this MRTK documentation elucidates the technical details of this feature:Hand tracking

    The setup is Unity 2020.3 LTS, Microsoft Mixed Reality OpenXR plugin, Unity OpenXR plugin, XR plugin management and XR Interaction Toolkit. No MRTK

    I noticed that you aren’t using MRTK yet. If you have any concerns in this way, could you please let us know the reason? We’d like to find out the best approach for you.

    ----
    If the response is helpful, please click "Accept Answer" and upvote it.
    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

    0 comments
  2. Xenotech Srl 1 Reputation point
    2021-11-22T14:40:47.777+00:00

    Hello Hernando,
    Thanks for your feedback.

    We're not using MRTK because we've been developing our internal framework for OpenXR development.
    We've already successfully implemented hand tracking in HL2 builds, the problem is only with holographic remoting in Unity. This is very frustrating, because the testing iteration, as of now, takes a very long time for even the smallest change in code. The holographic remoting tool for Unity would TERRIBLY improve our development experience with HL2.
    We also tried leveraging MRTK, but even using it in the Hand Interaction Example scene by Microsoft (Mixed Reality Toolkit Examples package) and activating both hand mesh & hand joint buttons, the hand tracking is still not working in the holographic remoting tool!
    Basically, using MRTK doesn't fix the holographic remoting issue for us.
    As i already mentioned, we've tried following all of your guides and double (even triple) checked all of the settings and package versions.
    All is working fine when building from VS to the device, but no hands in holographic remoring tool in Unity.
    Please let me know, this is very important for our HL2 development and future choices.
    Thanks!