Edit

High dynamic range (HDR) and low-light photo capture

  • Article

This article shows you how to use the AdvancedPhotoCapture class to capture high dynamic range (HDR) photos. This API also allows you to obtain a reference frame from the HDR capture before the processing of the final image is complete.

Other articles related to HDR capture include:

Note

Starting with Windows 10, version 1709, recording video and using AdvancedPhotoCapture concurrently is supported. This is not supported in previous versions. This change means that you can have a prepared LowLagMediaRecording and AdvancedPhotoCapture at the same time. You can start or stop video recording between calls to MediaCapture.PrepareAdvancedPhotoCaptureAsync and AdvancedPhotoCapture.FinishAsync. You can also call AdvancedPhotoCapture.CaptureAsync while video is recording. However, some AdvancedPhotoCapture scenarios, like capturing an HDR photo while recording video would cause some video frames to be altered by the HDR capture, resulting in a negative user experience. For this reason, the list of modes returned by the AdvancedPhotoControl.SupportedModes will be different while video is recording. You should check this value immediately after starting or stopping video recording to ensure that the desired mode is supported in the current video recording state.

Note

Starting with Windows 10, version 1709, when the AdvancedPhotoCapture is set to HDR mode, the setting of the FlashControl.Enabled property is ignored and the flash is never fired. For other capture modes, if the FlashControl.Enabled, it will override the AdvancedPhotoCapture settings and cause a normal photo to be captured with flash. If Auto is set to true, the AdvancedPhotoCapture may or may not use flash, depending on the camera driver's default behavior for the conditions in the current scene. On previous releases, the AdvancedPhotoCapture flash setting always overrides the FlashControl.Enabled setting.

Note

This article builds on concepts and code discussed in Basic photo, video, and audio capture with MediaCapture, which describes the steps for implementing basic photo and video capture. We recommend that you familiarize yourself with the basic media capture pattern in that article before moving on to more advanced capture scenarios. The code in this article assumes that your app already has an instance of MediaCapture that has been properly initialized.

There is a Universal Windows Sample demonstrating the use of the AdvancedPhotoCapture class that you can use to see the API used in context or as a starting point for your own app. For more information see, Camera Advanced Capture sample.

Advanced photo capture namespaces

The code examples in this article use APIs in the following namespaces in addition to the namespaces required for basic media capture.

using Windows.Media.Core;
using Windows.Media.Devices;

HDR photo capture

Determine if HDR photo capture is supported on the current device

The HDR capture technique described in this article is performed using the AdvancedPhotoCapture object. Not all devices support HDR capture with AdvancedPhotoCapture. Determine if the device on which your app is currently running supports the technique by getting the MediaCapture object's VideoDeviceController and then getting the AdvancedPhotoControl property. Check the video device controller's SupportedModes collection to see if it includes AdvancedPhotoMode.Hdr. If it does, HDR capture using AdvancedPhotoCapture is supported.

bool _hdrSupported;
private void IsHdrPhotoSupported()
{
    _hdrSupported = _mediaCapture.VideoDeviceController.AdvancedPhotoControl.SupportedModes.Contains(Windows.Media.Devices.AdvancedPhotoMode.Hdr);
}

Configure and prepare the AdvancedPhotoCapture object

Because you will need to access the AdvancedPhotoCapture instance from multiple places within your code, you should declare a member variable to hold the object.

private AdvancedPhotoCapture _advancedCapture;

In your app, after you have initialized the MediaCapture object, create an AdvancedPhotoCaptureSettings object and set the mode to AdvancedPhotoMode.Hdr. Call the AdvancedPhotoControl object's Configure method, passing in the AdvancedPhotoCaptureSettings object you created.

Call the MediaCapture object's PrepareAdvancedPhotoCaptureAsync, passing in an ImageEncodingProperties object specifying the type of encoding the capture should use. The ImageEncodingProperties class provides static methods for creating the image encodings that are supported by MediaCapture.

PrepareAdvancedPhotoCaptureAsync returns the AdvancedPhotoCapture object you will use to initiate photo capture. You can use this object to register handlers for the OptionalReferencePhotoCaptured and AllPhotosCaptured which are discussed later in this article.

if (_hdrSupported == false) return;

// Choose HDR mode
var settings = new AdvancedPhotoCaptureSettings { Mode = AdvancedPhotoMode.Hdr };

// Configure the mode
_mediaCapture.VideoDeviceController.AdvancedPhotoControl.Configure(settings);

// Prepare for an advanced capture
_advancedCapture = 
    await _mediaCapture.PrepareAdvancedPhotoCaptureAsync(ImageEncodingProperties.CreateUncompressed(MediaPixelFormat.Nv12));

// Register for events published by the AdvancedCapture
_advancedCapture.AllPhotosCaptured += AdvancedCapture_AllPhotosCaptured;
_advancedCapture.OptionalReferencePhotoCaptured += AdvancedCapture_OptionalReferencePhotoCaptured;

Capture an HDR photo

Capture an HDR photo by calling the AdvancedPhotoCapture object's CaptureAsync method. This method returns an AdvancedCapturedPhoto object that provides the captured photo in its Frame property.

try
{

    // Start capture, and pass the context object
    AdvancedCapturedPhoto advancedCapturedPhoto = await _advancedCapture.CaptureAsync();

    using (var frame = advancedCapturedPhoto.Frame)
    {
        // Read the current orientation of the camera and the capture time
        var photoOrientation = CameraRotationHelper.ConvertSimpleOrientationToPhotoOrientation(
            _rotationHelper.GetCameraCaptureOrientation());
        var fileName = String.Format("SimplePhoto_{0}_HDR.jpg", DateTime.Now.ToString("HHmmss"));
        await SaveCapturedFrameAsync(frame, fileName, photoOrientation);
    }
}
catch (Exception ex)
{
    Debug.WriteLine("Exception when taking an HDR photo: {0}", ex.ToString());
}

Most photography apps will want to encode a captured photo's rotation into the image file so that it can be displayed correctly by other apps and devices. This example shows the use of the helper class CameraRotationHelper to calculate the proper orientation for the file. This class is described and listed in full in the article Handle device orientation with MediaCapture.

The SaveCapturedFrameAsync helper method, which saves the image to disk, is discussed later in this article.

Get optional reference frame

The HDR process captures multiple frames and then composites them into a single image after all of the frames have been captured. You can get access to a frame after it is captured but before the entire HDR process is complete by handling the OptionalReferencePhotoCaptured event. You don't need to do this if you are only interested in the final HDR photo result.

Important

OptionalReferencePhotoCaptured is not raised on devices that support hardware HDR and therefore do not generate reference frames. Your app should handle the case where this event is not raised.

Because the reference frame arrives out of context of the call to CaptureAsync, a mechanism is provided to pass context information to the OptionalReferencePhotoCaptured handler. First you should call an object that will contain your context information. The name and contents of this object is up to you. This example defines an object that has members to track the file name and camera orientation of the capture.

public class MyAdvancedCaptureContextObject
{
    public string CaptureFileName;
    public PhotoOrientation CaptureOrientation;
}

Create a new instance of your context object, populate its members, and then pass it into the overload of CaptureAsync that accepts an object as a parameter.

// Read the current orientation of the camera and the capture time
var photoOrientation = CameraRotationHelper.ConvertSimpleOrientationToPhotoOrientation(
        _rotationHelper.GetCameraCaptureOrientation());
var fileName = String.Format("SimplePhoto_{0}_HDR.jpg", DateTime.Now.ToString("HHmmss"));

// Create a context object, to identify the capture in the OptionalReferencePhotoCaptured event
var context = new MyAdvancedCaptureContextObject()
{
    CaptureFileName = fileName,
    CaptureOrientation = photoOrientation
};

// Start capture, and pass the context object
AdvancedCapturedPhoto advancedCapturedPhoto = await _advancedCapture.CaptureAsync(context);

In the OptionalReferencePhotoCaptured event handler, cast the Context property of the OptionalReferencePhotoCapturedEventArgs object to your context object class. This example modifies the file name to distinguish the reference frame image from the final HDR image and then calls the SaveCapturedFrameAsync helper method to save the image.

private async void AdvancedCapture_OptionalReferencePhotoCaptured(AdvancedPhotoCapture sender, OptionalReferencePhotoCapturedEventArgs args)
{
    // Retrieve the context (i.e. what capture does this belong to?)
    var context = args.Context as MyAdvancedCaptureContextObject;

    // Remove "_HDR" from the name of the capture to create the name of the reference
    var referenceName = context.CaptureFileName.Replace("_HDR", "");

    using (var frame = args.Frame)
    {
        await SaveCapturedFrameAsync(frame, referenceName, context.CaptureOrientation);
    }
}

Receive a notification when all frames have been captured

The HDR photo capture has two steps. First, multiple frames are captured, and then the frames are processed into the final HDR image. You can't initiate another capture while the source HDR frames are still being captured, but you can initiate a capture after all of the frames have been captured but before the HDR post-processing is complete. The AllPhotosCaptured event is raised when the HDR captures are complete, letting you know that you can initiate another capture. A typical scenario is to disable your UI's capture button when HDR capture begins and then reenable it when AllPhotosCaptured is raised.

private void AdvancedCapture_AllPhotosCaptured(AdvancedPhotoCapture sender, object args)
{
    // Update UI to enable capture button
}

Clean up the AdvancedPhotoCapture object

When your app is done capturing, before disposing of the MediaCapture object, you should shut down the AdvancedPhotoCapture object by calling FinishAsync and setting your member variable to null.

await _advancedCapture.FinishAsync();
_advancedCapture = null;

Low-light photo capture

Starting with Windows 10, version 1607, AdvancedPhotoCapture can be used to capture photos using a built-in algorithm that enhances the quality of photos captured in low-light settings. When you use the low-light feature of the AdvancedPhotoCapture class, the system will evaluate the current scene and, if needed, apply an algorithm to compensate for low-light conditions. If the system determines that the algorithm is not needed, a regular capture is performed instead.

Before using low-light photo capture, determine if the device on which your app is currently running supports the technique by getting the MediaCapture object's VideoDeviceController and then getting the AdvancedPhotoControl property. Check the video device controller's SupportedModes collection to see if it includes AdvancedPhotoMode.LowLight. If it does, low-light capture using AdvancedPhotoCapture is supported.

bool _lowLightSupported;

_lowLightSupported = 
_mediaCapture.VideoDeviceController.AdvancedPhotoControl.SupportedModes.Contains(Windows.Media.Devices.AdvancedPhotoMode.LowLight);

Next, declare a member variable to store the AdvancedPhotoCapture object.

private AdvancedPhotoCapture _advancedCapture;

In your app, after you have initialized the MediaCapture object, create an AdvancedPhotoCaptureSettings object and set the mode to AdvancedPhotoMode.LowLight. Call the AdvancedPhotoControl object's Configure method, passing in the AdvancedPhotoCaptureSettings object you created.

Call the MediaCapture object's PrepareAdvancedPhotoCaptureAsync, passing in an ImageEncodingProperties object specifying the type of encoding the capture should use.

if (_lowLightSupported == false) return;

// Choose LowLight mode
var settings = new AdvancedPhotoCaptureSettings { Mode = AdvancedPhotoMode.LowLight };
_mediaCapture.VideoDeviceController.AdvancedPhotoControl.Configure(settings);

// Prepare for an advanced capture
_advancedCapture = 
    await _mediaCapture.PrepareAdvancedPhotoCaptureAsync(ImageEncodingProperties.CreateUncompressed(MediaPixelFormat.Nv12));

To capture a photo, call CaptureAsync.

AdvancedCapturedPhoto advancedCapturedPhoto = await _advancedCapture.CaptureAsync();
var photoOrientation = ConvertOrientationToPhotoOrientation(GetCameraOrientation());
var fileName = String.Format("SimplePhoto_{0}_LowLight.jpg", DateTime.Now.ToString("HHmmss"));
await SaveCapturedFrameAsync(advancedCapturedPhoto.Frame, fileName, photoOrientation);

Like the HDR example above, this example uses a helper class called CameraRotationHelper to determine the rotation value that should be encoded into the image so that it can be displayed properly by other apps and devices. This class is described and listed in full in the article Handle device orientation with MediaCapture.

The SaveCapturedFrameAsync helper method, which saves the image to disk, is discussed later in this article.

You can capture multiple low-light photos without reconfiguring the AdvancedPhotoCapture object, but when you are done capturing, you should call FinishAsync to clean up the object and associated resources.

await _advancedCapture.FinishAsync();
_advancedCapture = null;

Working with AdvancedCapturedPhoto objects

AdvancedPhotoCapture.CaptureAsync returns an AdvancedCapturedPhoto object representing the captured photo. This object exposes the Frame property which returns a CapturedFrame object representing the image. The OptionalReferencePhotoCaptured event also provides a CapturedFrame object in its event args. After you get an object of this type, there are a number of things you can do with it, including creating a SoftwareBitmap or saving the image to a file.

Get a SoftwareBitmap from a CapturedFrame

It's trivial to get a SoftwareBitmap from a CapturedFrame object by simply accessing the SoftwareBitmap property of the object. However, most encoding formats do not support SoftwareBitmap with AdvancedPhotoCapture, so you should check and make sure the property is not null before using it.

SoftwareBitmap bitmap;
if (advancedCapturedPhoto.Frame.SoftwareBitmap != null)
{
    bitmap = advancedCapturedPhoto.Frame.SoftwareBitmap;
}

In the current release, the only encoding format that supports SoftwareBitmap for AdvancedPhotoCapture is uncompressed NV12. So, if you want to use this feature, you must specify that encoding when you call PrepareAdvancedPhotoCaptureAsync.

_advancedCapture =
    await _mediaCapture.PrepareAdvancedPhotoCaptureAsync(ImageEncodingProperties.CreateUncompressed(MediaPixelFormat.Nv12));

Of course, you can always save the image to a file and then load the file into a SoftwareBitmap in a separate step. For more information about working with SoftwareBitmap, see Create, edit, and save bitmap images.

Save a CapturedFrame to a file

The CapturedFrame class implements the IInputStream interface, so it can be used as the input to a BitmapDecoder, and then a BitmapEncoder can be used to write the image data to disk.

In the following example, a new folder in the user's pictures library is created and a file is created within this folder. Note that your app will need to include the Pictures Library capability in your app manifest file in order to access this directory. A file stream is then opened to the specified file. Next, the BitmapDecoder.CreateAsync is called to create the decoder from the CapturedFrame. Then CreateForTranscodingAsync creates an encoder from the file stream and the decoder.

The next steps encode the orientation of the photo into the image file by using the BitmapProperties of the encoder. For more information about handling orientation when capturing images, see Handle device orientation with MediaCapture.

Finally, the image is written to the file with a call to FlushAsync.

private static async Task<StorageFile> SaveCapturedFrameAsync(CapturedFrame frame, string fileName, PhotoOrientation photoOrientation)
{
    var folder = await KnownFolders.PicturesLibrary.CreateFolderAsync("MyApp", CreationCollisionOption.OpenIfExists);
    var file = await folder.CreateFileAsync(fileName, CreationCollisionOption.GenerateUniqueName);

    using (var inputStream = frame)
    {
        using (var fileStream = await file.OpenAsync(FileAccessMode.ReadWrite))
        {
            var decoder = await BitmapDecoder.CreateAsync(inputStream);
            var encoder = await BitmapEncoder.CreateForTranscodingAsync(fileStream, decoder);
            var properties = new BitmapPropertySet {
                { "System.Photo.Orientation", new BitmapTypedValue(photoOrientation, PropertyType.UInt16) } };
            await encoder.BitmapProperties.SetPropertiesAsync(properties);
            await encoder.FlushAsync();
        }
    }
    return file;
}