Articulated Hand and Eye Tracking in Unity
HoloLens 2 has introduced some new and exciting capabilities, such as Articulated Hand and Eye Tracking.
The easiest way to leverage the new capability in Unity is through MRTK. There are also some example scenes to help you get started.
Building Blocks supporting Hands, eyes and others in MRTK
MRTK v2 provides a set of UI Controls and building blocks to help you accelerate your development.
 Button |
 Bounding Box |
 Manipulation Handler |
|---|---|---|
| A button control, which supports various input methods including HoloLens2's articulated hand | Standard UI for manipulating objects in 3D space | Script for manipulating objects with one or two hands |
 Slate |
 System Keyboard |
 Interactable |
| 2D style plane, which supports scrolling with articulated hand input | Example script of using the system keyboard in Unity | A script for making objects interactable with visual states and theme support |
 Solver |
 Object Collection |
 Tooltip |
| Various object positioning behaviors such as tag-along, body-lock, constant view size, and surface magnetism | Script for lay out an array of objects in a three-dimensional shape | Annotation UI with flexible anchor/pivot system, which can be used for labeling motion controllers and object. |
 App Bar |
 Pointers |
 Fingertip Visualization |
| UI for Bounding Box's manual activation | Learn about various types of pointers | Visual affordance on the fingertip, which improves the confidence for the direct interaction |
 Eye Tracking: Target Selection |
 Eye Tracking: Navigation |
|
| Combine eyes, voice, and hand input to quickly and effortlessly select holograms across your scene | Learn how to auto scroll text or zoom into focused content based on what you're looking at | Examples for logging, loading, and visualizing what users have been looking at in your app |
Example Scenes
Explore MRTK's various types of interactions and UI controls in this example scene.
You can find other example scenes in Mixed Reality Toolkit GitHub under the Assets/MixedRealityToolkit.Examples/Demos folder.
Next Development Checkpoint
If you're following the Unity development journey we've laid out, you're in the midst of exploring the MRTK core building blocks. From here, you can continue to the next building block:
Or jump to Mixed Reality platform capabilities and APIs:
You can always go back to the Unity development checkpoints at any time.