Muokkaa

Jaa


Camera setup in Unity

When you wear a mixed reality headset, it becomes the center of your holographic world. The Unity Camera component will automatically handle stereoscopic rendering and follow your head movement and rotation. However, to fully optimize visual quality and hologram stability, you should set the camera settings described below.

HoloLens vs VR immersive headsets

The default settings on the Unity Camera component are for traditional 3D applications, which need a skybox-like background as they don't have a real world.

  • When running on an immersive headset, you're rendering everything the user sees, and so you'll likely want to keep the skybox.
  • However, when running on a holographic headset like HoloLens, the real world should appear behind everything the camera renders. Set the camera background to be transparent (in HoloLens, black renders as transparent) instead of a Skybox texture:
    1. Select the Main Camera in the Hierarchy panel
    2. In the Inspector panel, find the Camera component and change the Clear Flags dropdown from Skybox to Solid Color
    3. Select the Background color picker and change the RGBA values to (0, 0, 0, 0)
      1. If setting this from code, you can use Unity's Color.clear

MRTK will handle specific camera settings automatically, based on the configuration in the camera system profile.

Namespace: Microsoft.MixedReality.Toolkit.CameraSystem
Type: MixedRealityCameraSystem

To check the camera's opaqueness, the MixedRealityCamera system has an IsOpaque property.

CoreServices.CameraSystem.IsOpaque;

Camera setup

Whatever kind of experience you're developing, the Main Camera is always the primary stereo rendering component attached to your device's head-mounted display. It'll be easier to lay out your app if you imagine the starting position of the user as (X: 0, Y: 0, Z: 0). Since the Main Camera is tracking movement of the user's head, the starting position of the user can be set by setting the starting position of the Main Camera.

The central choice you need to make is whether you're developing for HoloLens or VR immersive headsets. Once you've got that, skip to whichever setup section applies.

HoloLens camera setup

For HoloLens apps, you need to use anchors for any objects you want to lock to the scene environment. We recommend using unbounded space to maximize stability and create anchors in multiple rooms.

Follow this step-by-step tutorial to add and automatically configure Mixed Reality Toolkit in your Unity project. It's also possible to work directly with the MixedRealityPlayspace class from MRTK for Unity and set the Target Scale to World:

MRTK settings window

MRTK should handle the position of the playspace and camera automatically, but it's good to double check:

MRTK playspace

  1. From the Hierarchy panel, expand the MixedRealityPlayspace GameObject and find the Main Camera child object
  2. In the Inspector panel, find the Transform component and change the Position to (X: 0, Y: 0, Z: 0)

VR camera setup

Windows Mixed Reality supports apps across a wide range of experience scales, from orientation-only and seated-scale apps up through room-scale apps. On HoloLens, you can go further and build world-scale apps that let users walk beyond 5 meters, exploring an entire floor of a building and beyond.

Your first step in building a mixed reality experience in Unity is to determine which experience scale your app will target:

Room-scale or standing experiences

Note

If you're building for HL2, we recommend creating an eye-level experience, or consider using Scene Understanding to reason about the floor of your scene.

Use the MixedRealityPlayspace class from MRTK for Unity and set the Target Scale to either Room or Standing:

MRTK settings window

MRTK should handle the position of the playspace and camera automatically, but it's good to double check:

MRTK playspace

  1. From the Hierarchy panel, expand the MixedRealityPlayspace GameObject and find the Main Camera child object
  2. In the Inspector panel, find the Transform component and change the Position to (X: 0, Y: 0, Z: 0)

Seated experiences

Use the MixedRealityPlayspace class from MRTK for Unity and set the Target Scale to Seated:

MRTK settings window

MRTK should handle the position of the playspace and camera automatically, but it's good to double check:

MRTK playspace

  1. From the Hierarchy panel, expand the MixedRealityPlayspace GameObject and find the Main Camera child object
  2. In the Inspector panel, find the Transform component and change the Position to (X: 0, Y: 0, Z: 0)

Setting up the camera background

If you're using MRTK, the camera's background is automatically configured and managed. For XR SDK or Legacy WSA projects, we recommend setting the camera's background to solid black on HoloLens and keeping the skybox for VR.

Using multiple cameras

When there are multiple Camera components in the scene, Unity knows which camera to use for stereoscopic rendering based on which GameObject has the MainCamera tag. In legacy XR, it also uses this tag to sync head tracking. In XR SDK, head tracking is driven by a TrackedPoseDriver script attached to the camera.

Sharing depth buffers

Sharing your app's depth buffer to Windows each frame will give your app one of two boosts in hologram stability, based on the type of headset you're rendering for:

  • VR immersive headsets can take care of positional reprojection when a depth buffer is provided, adjusting your holograms for misprediction in both position and orientation.
  • HoloLens headsets have a few different methods. HoloLens 1 will automatically select a focus point when a depth buffer is provided, optimizing hologram stability along the plane that intersects the most content. HoloLens 2 will stabilize content using Depth LSR (see Remarks).

MRTK's configuration dialog will attempt to set depth buffer settings for both XR SDK and legacy WSA, but it's good to check those tabs and verify the settings in Unity.

Using clipping planes

Rendering content too close to the user can be uncomfortable in mixed reality. You can adjust the near and far clip planes on the Camera component.

  1. Select the Main Camera in the Hierarchy panel
  2. In the Inspector panel, find the Camera component Clipping Planes and change the Near textbox from 0.3 to 0.85. Content rendered even closer can lead to user discomfort and should be avoided per the render distance guidelines.

Recentering the camera

If you're building a seated-scale experience, you can recenter Unity's world origin at the user's current head position by calling the XR.InputTracking.Recenter method in legacy XR or the XRInputSubsystem.TryRecenter method in XR SDK.

Teleportation

This feature is typically reserved for VR experiences:

MRTK provides an in-box teleport system which automatically works across articulated hands and controllers.

Reprojection modes

Both HoloLens and immersive headsets will reproject each frame your app renders to adjust for any misprediction of the user's actual head position when photons are emitted.

By default:

  • VR immersive headsets will take care of positional reprojection if the app provides a depth buffer for a given frame. Immersive headsets will also adjust your holograms for misprediction in both position and orientation. If a depth buffer isn't provided, the system will only correct mispredictions in orientation.
  • Holographic headsets like HoloLens 2 will take care of positional reprojection whether the app provides its depth buffer or not. Positional reprojection is possible without depth buffers on HoloLens as rendering is often sparse with a stable background provided by the real world.

MRTK doesn't currently have helpers for the reprojection mode. Please see one of the other tabs for more information.

Next Development Checkpoint

If you're following the Unity development journey we've laid out, you're in the midst of exploring the MRTK core building blocks. From here, you can continue to the next building block:

Or jump to Mixed Reality platform capabilities and APIs:

You can always go back to the Unity development checkpoints at any time.

See also