Eye tracking is not responding within apps that use eye tracking with HoloLens2

yuya aikawa 1 Reputation point
2022-11-28T10:32:08.617+00:00

Problem:
I have developed an app that uses eye tracking with HoloLens2 (the number in the upper left corner counts up when you gaze input against the Dwell button), but the eye tracking is not responding.

264745-%E3%82%B9%E3%82%AF%E3%83%AA%E3%83%BC%E3%83%B3%E3%82%B7%E3%83%A7%E3%83%83%E3%83%88-2022-11-28-174013.png

What I want to:

  • Enable gaze input
  • Gaze cursor out

What we tried:

The following two things are known at this time.

  • It works fine on the Unity Editor, but not on the actual device.
  • The eye tracking samples in the MRTK Examples Hub can be used on the actual device.

I believe that there is no mistake in the object placement or scripts, as they work fine on the Unity Editor.
Also, the eye tracking in the MRTK Examples Hub works fine, so I am wondering if there are any mistakes in the settings (calibration, etc.) on the HoloLens2 side.

As a possible cause, I thought it might be an incorrect setting for the actual device when building the application, so I scoured the official documentation and found that the following three things are required, all of which have been addressed.

Eye tracking requirements checklist

For eye tracking to work correctly, the following requirements must be
met. If you are new to eye tracking on HoloLens 2 and to how eye
tracking is set up in MRTK, don't worry! We will go into detail on how
to address each of them further below.

  1. An 'Eye Gaze Data Provider' must be added to the input system. This provides eye tracking data from the platform.
  2. The 'GazeInput' capability must be enabled in the application manifest. This capability can be set in Unity 2019, but in Unity
    2018 and earlier this capability is only available in Visual Studio
    and through the MRTK build tool
  3. The HoloLens must be eye calibrated for the current user. Check out our sample for detecting whether a user is eye calibrated or
    not
    .

Also strange, but I can input eye gaze within the application about 1/20th of a turn.
So I thought maybe the angle to attach the HMD or the tightness of the HMD was weak, but that seems to be wrong too. (Because I tried the Example Hub with the same way of attaching it and it works fine this way.)

To be honest, I'm stuck, not knowing what the cause is.
If you have any ideas, even if it's not a direct solution, I would love to hear from you.

Thank you in advance.

version:
Unity 2020.3.40f1(LTS)
MRTK2.8.0

HoloLens Development
HoloLens Development
HoloLens: A family of Microsoft self-contained, holographic devices that enable engagement with digital content and interaction with holograms in the surrounding environment.Development: The process of researching, productizing, and refining new or existing technologies.
390 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Zuocheng Wang - MSFT 3,086 Reputation points Microsoft Vendor
    2022-11-29T08:36:46.537+00:00

    Hello, Welcome to Microsoft Q&A,

    You can troubleshoot in the following ways.

    1. For the project, how do you configure the MixedRealityToolkitProfile? For testing you can directly use the EyeTrackingDemoConfigurationProfile in MixedRealityToolkitExamples/Demos-EyeTracking. You can also refer to this profile and Eye Tracking Basic Setup - MRTK 2 | Microsoft Learn to configure your own profile.
    2. For deployment, please also check whether Eye Gaze Interaction Profile is added in XR Plug-in Managment->OpenXR->UWP->Interaction Profiles.
    3. For debugging in Unity, are you running directly in Unity or remotely debugging through Remote for Play Mode? If you are running directly in Unity, then it needs to be clarified that the default is to use HeadGaze instead of EyeGaze.
    4. For the device, please also check whether you have given this application permission in the device's Settings->Privacy->AppPermissions->Eye tracker.

    ----
    If the response is helpful, please click "Accept Answer" and upvote it.
    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

    0 comments No comments