How to stream video to HoloLens 2 with low latency?

Fabian 6 Reputation points


I am looking for a way to stream video (without audio) to HoloLens 2 with a latency < 300ms?
I'd like to create a resizable virtual window on the HoloLens that shows the video streams.

So far I tried:

  1. "HTTP Live Streaming" (HLS) => latency too high
  2. WebRTC => not supported in Edge and Firefox Reality browser
  3. MP4 Progressive Download ("Pseudo-Streaming") => latency too high
  4. RTSP + RTP (over UDP)
    4a. with VLC Player app => latency >1s even with :network-cache=0 etc.
    4b. with Huczeq's "RTSP Player" app => best latency, but still approx 700ms

My RTSP/RTP server (gst-rtsp-server) is already properly configured for near zero latency.
I get < 150ms latency when streaming to gstreamer's rtspsrc module on another computer over wifi.

Does the HoloLens 2 provide another way to receive and show a video stream with low latency?

I haven't tested yet whether HoloLens' browser supports the Media Source Extension API (MSE).

HoloLens Development
HoloLens Development
HoloLens: A family of Microsoft self-contained, holographic devices that enable engagement with digital content and interaction with holograms in the surrounding environment.Development: The process of researching, productizing, and refining new or existing technologies.
390 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Hernando Ren 2,171 Reputation points

    Hello, Welcome to Microsoft Q&A,

    MixedReality-WebRTC can help you to integrate peer-to-peer real-time video communication into your application, which can be used outside Web Browser. It is recommended you start with this tutorial to create a simple unity application to receive remote media. For more information please refer to:MixedReality-WebRTC

    If the response is helpful, please click "Accept Answer" and upvote it.
    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

    1 person found this answer helpful.