Connection between HoloLens 2 and Windows Laptop

Question

I want to develop an app for HoloLens 2 and an app for Windows Laptop.

Desired features:

1. HoloLens 2 can scan a QR Code on the Windows Laptop screen and establish a peer to peer connection. (Maybe through WebRTC, I'm not sure.)
2. Windows Laptop can send .obj or .gltf 3D model through the connection.
3. HoloLens 2 will send its "Head 6dof", "Hand data" and "Eye gaze data" in real time.

Questions:

1. Is it better for me to develop this app using Unity or Visual Studio only?
2. Is there any useful package/repository which can make my development journey easier?
3. Is there any method to implement the multi-user feature in local network situation? (I've seen the document made by Microsoft talking about Photon Unity Networking (PUN) which needs an internet connection)

Thanks for your help in advance!

Answer 1

Hello, Welcome to Microsoft Q&A,

To develop the desired application, Unity and Visual Studio will be your primary tools. It’s recommended to use Unity 2021.3 LTS and Visual Studio 2022.

For information on using QR code tracking, please refer to QR code tracking overview - Mixed Reality | Microsoft Learn.

Currently, there’s no official solution for Web-RTC feature. You can use Media Capture API to implement real-time audio and video communication.

To get head, hand and eye tracking data, you can refer to the following links:

https://learn.microsoft.com/en-us/windows/mixed-reality/mrtk-unity/mrtk2/features/input/gaze?view=mrtkunity-2022-05 - how-get-the-current-gaze-target

Eye Tracking Eye Gaze Provider - MRTK 2 | Microsoft Learn

IMixedRealityHand Interface (Microsoft.MixedReality.Toolkit.Input) | Microsoft Learn  

For the issue on implementing the multi-user feature over a local network, you can refer to Is it possible to create a multi-user app for HoloLens without internet connection (offline mode) ? - Microsoft Q&A.

---

If the response is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.