QR code tracking overview

Article
05/24/2024

To access the GitHub repo for the QR code sample:

With Windows Mixed Reality and HoloLens headsets, your app can detect QR codes in the environment around the headset, establishing a coordinate system at each code's real-world location. You can also render holograms in the same location on multiple devices to create a shared experience. Once you enable your device's webcam, you'll recognize QR codes in the latest versions of your projects. For best results going into production, we recommend that you review the best practices section.

In this article, you'll learn about:

Supported devices for QR code tracking
Supported QR code versions
Best practices for QR code detection
Troubleshooting and FAQ

Device support

Product	HoloLens 1st Gen	HoloLens 2	Immersive headsets
OpenXR		✔️ (with OpenXR Runtime version 113 and OpenXR XR_MSFT_scene_marker extension)
Mixed Reality Unity Plugin		✔️ (with OpenXR Runtime version 113 and Mixed Reality Unity Plugin `ARMarkerManager`)
Legacy QR SDK	✔️	✔️	✔️

Note

The following is specific to the legacy QR code APIs. We recommend using the OpenXR implementation for QR codes instead.

Using the legacy QR code SDK, QR code tracking with immersive Windows Mixed Reality headsets on desktop PCs is supported on Windows 10 Version 2004 and higher. Use the Microsoft.MixedReality.QRCodeWatcher.IsSupported() API to determine whether the feature is supported on the current device.

Supported QR code versions

The following table lists the supported and unsupported QR code versions:

	QR code versions
Supported:	QR Code versions 1 - 10 Micro QR Codes M1-M4 All encodings
Not supported:	QR Codes 10 - 20 (not fully supported) QR Codes versions 21+ Tweaked modules, for example: circular modules, reflections, inverted (black/white). Logos Structured appends

Best practices for QR code detection

Quiet zones around QR codes

To be read correctly, QR codes require a margin around all sides of the code. This margin must not contain any printed content and should be four modules (a single black square in the code) wide.

The official QR code documentation contains more information about quiet zones.

Lighting and backdrop

QR code detection quality is susceptible to varying illumination and backdrop.

Under normal lighting conditions, provide enough contrast for the black/white modules for better performance.
In extreme lighting conditions with bright lighting or dark backdrop, you can try to reduce and adjust contrast which could improve QR code detection rate. The white background within the QR code can be reduced from 255 downwards.

Size of QR codes

Windows Mixed Reality and HoloLens devices don't work with QR codes with sides smaller than 5 cm.

For QR codes with sides ranging from 5 cm to 10 cm, the device must be fairly close to detect the code. It may take longer than average to detect the code.

The exact time to detect codes depends not only on the size of the QR codes, but how far away you are from the code. Moving closer to the code will help offset issues with size.

Distance and angular position from the QR code

The tracking cameras can only detect a certain level of detail. For small codes (less than 10 cm along the sides), you must be fairly close. For a version 1 QR code varying from 10 cm to 25 cm in size, the minimum detection distance is between 0.15 meters and 0.5 meters.

The detection distance for size increases linearly, but also depends on supported QR version or module size. The higher the version, the smaller the modules, which can only be detected from a closer position. You can also try micro-QR codes if you want the distance of detection to be longer. QR detection works with a range of angles += 45 deg to ensure we have proper resolution to detect the code.

Other detection considerations:

QR codes on curved surfaces aren’t supported.
In-plane orientation is supported. Out of plane should be <= +-45 looking straight on to have better detection.
The physical size of the QR code should have modules of at least 2/3 pixels. Note: Higher versions of QR codes will have smaller modules.

For the tradeoff relationship between distance and size of the QR code for optimal detection, see the following graph:

Important

Always make sure you have enough contrast and a proper border.

Managing QR code data

Windows Mixed Reality and HoloLens devices detect QR codes at the system level in the driver. When the device is rebooted or the driver restarts, the detected QR codes history is cleared. QR codes redetected are treated as new objects.

We recommend configuring your app to ignore QR codes older than a specific timestamp, which can be specified within the app. The QR Code API specifies the time that last detection happened. Most app developers will use the system time when the app is launched to determine the time a QR code is detected.

QR code data aren’t app-specific. Upon app launch, there will be a list of available QR codes being provided. The app developer will determine which QR codes are relevant to this app.

QR code placement in a space

For recommendations on where and how to place QR codes, refer to Environment considerations for HoloLens.

Troubleshooting and FAQ

What capabilities are needed?

To enable QR code tracking in your HoloLens application, add the webcam capability to the application’s manifest. If you are developing in Unity, you can configure this from the player settings.

Additionally, users may be prompted by the permissions dialog to grant your app webcam permissions. This only happens once over the lifetime of an app. In cases such as the app explicitly requesting webcam access, the dialog will not reappear.

For Unity apps with the WebCam capability, enabling ARMarkerManager in a scene can trigger the webcam permission dialog to appear.

For native OpenXR C++ apps, initial calls to xrComputeNewSceneMSFT with XrNewSceneComputeInfoMSFT::requestedFeatures containing XR_SCENE_COMPUTE_FEATURE_MARKER_MSFT can trigger the permissions dialog.

Note

The following is specific to the legacy QR code APIs. We recommend using the OpenXR implementation for QR codes instead.

You'll need the webcam capability added to your manifest (checkbox in Unity Capabilities). If you're building as a standard UWP project, it’s also in the package.appxmanifest in the solution project.

In the legacy QR code SDK, request access by calling the following method:

#if WINDOWS_UWP

async QRCodeWatcher.RequestAccessAsync();

#endif

The access status should be (status == QRCodeWatcherAccessStatus::Allowed).

If access is denied, then the feature will return access denied when you start the QRTracking.

This API should be called before you construct the QRCodeWatcher object.

If you're running the project from Unity, you also need to ensure that you're calling from the UI thread. Otherwise, the API will always return denied. For more information, see AppCallbacks class in the Unity Manual.

For more information on setting up your project for Unity, see configure Unity for Windows Mixed Reality.

How do I make the QR Code Tracking Feature work on HoloLens 2 devices?

QR tracking is automatic on HoloLens 2, and you'll need the "webcam" capability added to your app.

Where do I find the API plugin files?

Applications should use the HoloLen’s OpenXR Runtime for detecting QR Codes. OpenXR for Windows Mixed Reality (113.2403.5001 or greater) provides QR code support on HoloLens devices.

For Unity apps, QR code support is provided through the ARMarkerManager component provided with Mixed Reality OpenXR Plugin.

For QR code support in non-Unity apps, use the OpenXR XR_MSFT_scene_marker extension.

Note

The following is specific to the legacy QR code APIs. We recommend using the OpenXR implementation for QR codes instead.

All the required files and documentation can be found here on NuGet here:

Microsoft.MixedReality.QR

How do I prepare a Unity app to use ARMarkerManager to detect QR codes?

In Unity apps, the ARMarkerManager component is used to handle how QR code detection within a physical environment is represented in a scene. ARMarkerManager provides:

Events for notifying subscribers when a detected QR code has been added to the manager, updated, or removed.
A collection of ARMarker trackables for QR codes under detection.
Methods returning data for QR codes as specified by trackable ID.

To use ARMarkerManager in your app, you need to import the Mixed Reality OpenXR Plugin package.

To import the package:

Download and run the Mixed Reality Feature Tool.
Install the OpenXR plugin.

See Welcome to the Mixed Reality Feature Tool for detailed instructions on how to use the tool.

The general procedure next is to:

In your Unity project, enable WebCam capabilities.
Create a prefab and attach the ARMarker component as shown:
Open a scene you want QR codes to be detected when running.
Attach ARMarkerManager to a GameObject in your scene and set Marker Prefab to the prefab you created.

For more detailed instructions and information, go to QR codes in Unity.

For an example of how to use QR codes in your Unity app, see the QR code sample scenario on GitHub.

How do I prepare a non-Unity app to use OpenXR to detect QR codes?

QR code detection in OpenXR is provided through the XR_MSFT_scene_marker extension. This extension allows multiple QR codes to be tracked and can provide size, position, last detection timestamp, UUID, and buffered QR code data for detected QR codes.

To enable QR code support, both the XR_MSFT_scene_marker and XR_MSFT_scene_understanding extensions must be enabled.

For a detailed step-by-step, see the XR_MSFT_scene_marker extension overview in the OpenXR 1.0 specification.

For an example of how to use QR codes, see Scene_QRCode.cpp in the SceneUnderstandingUwp sample on GitHub.

How do I prepare a UWP to use Microsoft.MixedReality.QR.QRCodeWatcher?

Note

The following is specific to the legacy QR code APIs. We recommend using the OpenXR implementation for QR codes instead.

Use the NuGet pack to unpack the required files.
Add a reference to the Microsoft.MixedReality.QR.winmd in your project and start using the API.
Add the correct architecture versions of the plugins and use them accordingly in the build.

How do I prepare Unity with the Microsoft.MixedReality.QR.QRCodeWatcher?

Note

The following is specific to the legacy QR code APIs. We recommend using the OpenXR implementation for QR codes instead.

Use NuGet for Unity and point to the NuGet pack above.

How can I make QR codes?

Check out any QR code generator.

If QR code tracking doesn't work in general, what do I do?

Is the QR Code version a supported version? We don't support the high-density versions like version 40. Nothing above version 10 is guaranteed; versions above 20 aren’t supported.
Are you close enough to the QR code? See distance and angular position from the QR code.
How is the lighting? There’s a known issue where detection becomes difficult when the QR code is on a dark environmental background--the QR code will appear washed out to the camera due to high contrast. For more information, see lighting and backdrop.

What's the accuracy?

When detected in a single frame, the size is expected to have at most a 1% error from the actual size. For example, a 10 cm code might be up to +/- 1 mm off in measured size. Under continuous detection, a code's position might drift around by up to +/- 2.5 mm. Once you’ve moved out of detection range, a previous detection's position is up to the mercy of the map error.

How close do I need to be to the QR code to detect it?

The distance obviously depends on the size of the QR code, and also what version it is. For more information, see distance and angular position from the QR code.

On HoloLens 2, for a version 1 QR code varying from 5 cm sides to 25 cm sides, the minimum detection distance ranges from 0.25 meters to 0.5 meters. The furthest these can be detected from goes from about 0.5 m for the smallest code to two meters for the bigger.
On Windows Mixed Reality, these distances for the sizes are halved.
For any codes bigger, extrapolate--the detection distance for size increases linearly. For any code smaller, detection simply won’t occur--4-5 cm is the smallest we can detect.

Why can't I read QR codes with logos?

QR codes with logos are not supported.

The QR codes are detected, so why am I getting no data?

If the platform can’t decode the QR code, there will be no data. You can use the stream and interpret the data using open-source code.
Some features, such as structure append, aren’t supported.

For more information, see what QR code versions are supported?.

Are QR codes saved at the ‘space’ level or app level?

QR codes are saved at the system level in driver session, or boot session on HoloLens. For more information, see managing QR code data.

How does that work with the underlying platform? Where do they persist?

Detected QR codes are persisted in memory by the driver.

How do I debug the app in Visual Studio when I receive the "`Microsoft.MixedReality.QR.pdb` not found" error message?

Note

The following is specific to the legacy QR code APIs. We recommend using the OpenXR implementation for QR codes instead.

Symbol loading for Microsoft.MixedReality.QR.dll was skipped because it isn’t specified in the included modules list.

To reproduce

Note

The following is specific to the legacy QR code APIs. We recommend using the OpenXR implementation for QR codes instead.

Follow the steps to reproduce this behavior:

Install Microsoft.MixedReality.QR (NuGet) and MRTK
Try to debug

You should be able to debug the app with Microsoft.MixedReality.QR.dll, but the DLL isn’t found:

Microsoft.MixedReality.QR.pdb not found error message

Is there a limit of how many QR codes can be scanned in a session?

There is a limit of 100 QR codes saved at the system level in the driver, within 10 minutes.

Has QR code smoothing process changed?

QR code smoothing process changed after 20H2, from pose smoothing to size smoothing. To revert behavior to the previous smoothing process, one would apply a smoothing algorithm if the new QR code pose data is detected within 5 seconds or 10 cm of the previously detected frame. The transform data is blended via a weighted ratio of 90% of the current frame with 10% of the previous frame.

Share via

QR code tracking overview

Device support

Supported QR code versions