Edit

Architecture overview — MRTK3

  • Article

Architecture MRTK3

One of the goals with MRTK3 was to take everything we've learned from the start of MRTK2 back in early 2018, combine it with the work that's been done by our industry partners across OpenXR and Unity since then, and come out the other side with a strong, extensible foundation that allows MRTK to focus more on providing differentiators and an overall improved user (and developer!) experience.

Input and interactions

The overall architecture of the input stack of MRTK3 is built on four foundational components:

  1. OpenXR
    1. Khronos Specification
    2. Unity OpenXR Plugin documentation
    3. Mixed Reality OpenXR Plugin documentation
  2. Unity subsystems
  3. Unity's Input System
  4. Unity's XR Interaction Toolkit

along with a layer of MRTK-defined interactors and subsystems, providing features like poke and speech.

OpenXR

OpenXR is the interface between an application and an XR runtime system, allowing for a common set of features to be called generically and allow the hardware-specific implementation to be handled by the XR runtime. Adopting this in MRTK3, along with Unity's Input System, lets Unity handle more of the cross-platform and extensible input story while allowing MRTK3 to focus on helping you build rich experiences on top.

Subsystems

Subsystems and Unity's SubsystemManager should be conceptually familiar to MRTK2 users, as they're the new "data providers". The idea is that different platforms or services can provide an implementation of a specific type of MRTK subsystem and have that run when relevant, providing data to MRTK3 and the app overall, just like data providers did to the various systems in MRTK2. Since we're focusing on OpenXR, the goal is that many features are covered by a cross-vendor EXT extension in OpenXR and multiple subsystems aren't needed, but vendor-specific extensions can represent early tech advancements that we want to support.

Unity Input System

Conceptually, Unity's Input System will also feel familiar to users of the MRTK2 controller mapping profile. It provides a central place for mapping the buttons and other input axes on a controller or hand to a set of actions. These actions are then consumed by Unity's XR Interaction Toolkit (XRI) and MRTK3, so the object being interacted with doesn't care as much about what is manipulating it, just that it is.

XR Interaction Toolkit

XRI provides a foundation of interactors and interactables. MRTK builds upon this with its own set of interactors and interactables, allowing for additional features like articulated hand tracking, gaze, and pinch.

See also