How to check via code the implemented sampling rate in Hololens 2 with MRTK3

penguin 0 Reputation points
2024-03-05T06:16:06.7066667+00:00

Inquiries

  1. May I see how to check via code the implemented sampling rate with MRTK3?
  2. Is the origin coordinate via FuzzyGazeInteractor (RayOriginTransform) the average of both eye gaze coordinates?

Problem statement

The origin coordinates of eye gaze ray via FuzzyGazeInteractor have unique values on every Unity frame (approx. 60 fps; See the first image; delta time, frame rate, fallback, origin coordinates of head gaze, and that of eye gaze; the units are sec and m). However, the default sampling rate is likely 30 Hz ("Available eye tracking data" in https://learn.microsoft.com/en-us/windows/mixed-reality/design/eye-tracking). It was expected that this implemented sampling rate is equal to or above 60 fps to have unique values.

Background

The coordinates of eye gaze via FuzzyGazeInteractor (attached to GazeInteractor gameobject) fall back to that of head gaze (Row 14 to 16 in the first image) and not (Row 17 to 22) as expected. However, it has much less pair of the same coordinates in eye gaze data (60 fps vs. 30 fps). Hence, it was expected that the sampling rate of eye gaze is equal to or more than 60 fps. This seemed likely, as the sampling rate seems modifiable to 60 or 90 Hz via extended eye tracking APIs ("Extended Eye Tracking data" in https://learn.microsoft.com/en-us/windows/mixed-reality/design/eye-tracking). However, after following steps to install Microsoft.MixedReality.EyeTracking and to incorporate ExtendedEyeGazeDataProvider.cs (https://learn.microsoft.com/en-us/windows/mixed-reality/develop/unity/extended-eye-tracking-unity), coordinates still had no alignment between that via FuzzyGazeInteractor and that via extended eye tracking APIs (See the second image; all 30, 60, 90 Hz tried; combined origin and disparity columns added; the combined origin had the average of the coordinates of both eyes). Besides, not in images here but, the incorporation of the extended APIs have now failed the fallback function from eye gaze to head gaze in a large time span (occlusion by eyelids but no fallback shift). Hence, it seems reasonable to stick to FuzzyGazeInteractor for eye tracking. But I am not sure of how to check the sampling rate via coding, along with how the origin coordinate derives.

Environments

Hololens 2

Windows 10, Visual Studio 2022 with workloads (https://learn.microsoft.com/en-us/windows/mixed-reality/develop/install-the-tools)

Unity 2022.3.19f1 with MRTK3 (https://learn.microsoft.com/en-us/training/modules/learn-mrtk-tutorials/)

User's image

User's image

HoloLens Development
HoloLens Development
HoloLens: A family of Microsoft self-contained, holographic devices that enable engagement with digital content and interaction with holograms in the surrounding environment.Development: The process of researching, productizing, and refining new or existing technologies.
380 questions
{count} votes