Edit

MFSampleExtension_VideoEncodeD3D12ReconstructedPicture attribute

Contains the reconstructed picture from a Direct3D 12 video encoder, provided as an IMFMediaBuffer.

Data type

An IMFMediaBuffer.

Remarks

The value of this attribute is a reconstructed picture provided by a video encoder MFT that uses Direct3D 12. The reconstructed picture is equivalent to the frame that would be generated by a video decoder. The encoder typically generates these pictures internally for use as reference frames when encoding future frames.

The reconstructed picture is an ID3D12Resource wrapped in an IMFMediaBuffer. The ID3D12Resource contains a picture described by D3D12_RESOURCE_DESC, which can be accessed through ID3D12Resource::GetDesc. The returned resource has the same dimensions, with alignment, as the Direct3D 12 input surface to the encoder. The alignment is typically 16 pixels due to hardware constraints. If the actual width and height are smaller than the aligned dimensions, the application should interpret the top-left rectangular region as the active region.

Use IMFAttributes::SetUnknown to attach an IMFMediaBuffer containing the reconstructed picture to an output sample. Use IMFAttributes::GetUnknown to retrieve the IMFMediaBuffer containing the reconstructed picture from an output sample.

The reconstructed picture is only generated when the entire frame has been encoded. If the encoder processes the frame in multiple slices, the reconstructed picture should be attached to the IMFSample of the final slice.

To enable reconstructed picture output, set the CODECAPI_AVEncVideoD3D12ReconstructedPictureOutputMode property to a non-zero value.

Examples

The following example shows a helper function that can be used by implementations of a D3D12-based encoder MFT. The function shows how to attach the reconstructed frame (ID3D12Resource) onto the output sample. This assumes that each output sample contains a complete frame. If the encoder uses multiple slices, the reconstructed picture should be attached only to the IMFSample of the last slice.

#include <mfapi.h>
#include <mfobjects.h>
#include <codecapi.h>
#include <wil/com.h>
#include <wil/resource.h>
#include <mfd3d12.h>

// Function to attach the D3D12 resource of a reconstructed frame to the output sample.
HRESULT AttachD3D12ResourceToMediaBuffer(
    _In_ ID3D12Resource* reconstructedPictureD3D12Resource,
    _In_ ID3D12CommandQueue* synchronizationObjectQueue,
    _In_ IMFSample* outputSample)
{
    wil::com_ptr_nothrow<IMFMediaBuffer> mediaBuffer;
    RETURN_IF_FAILED(MFCreateDXGISurfaceBuffer(__uuidof(ID3D12Resource),
        reconstructedPictureD3D12Resource, 0, FALSE, &mediaBuffer));

    wil::com_ptr_nothrow<IMFD3D12SynchronizationObjectCommands> outputSynchronizationObject;
    wil::com_ptr_nothrow<IMFDXGIBuffer> dxgiBuffer;
    RETURN_IF_FAILED(mediaBuffer.query_to(&dxgiBuffer));

    RETURN_IF_FAILED(dxgiBuffer->GetUnknown(MF_D3D12_SYNCHRONIZATION_OBJECT,
        IID_PPV_ARGS(&outputSynchronizationObject)));

    RETURN_IF_FAILED(
        outputSynchronizationObject->EnqueueResourceReady(synchronizationObjectQueue));

    return outputSample->SetUnknown(
        MFSampleExtension_VideoEncodeD3D12ReconstructedPicture, mediaBuffer.get());
}

The following example shows how an app can retrieve the reconstructed picture (ID3D12Resource) from the IMFSample objects that are produced by a Direct3D 12 encoder MFT.

#include <mfapi.h>
#include <mfobjects.h>
#include <codecapi.h>
#include <wil/com.h>
#include <wil/resource.h>
#include <d3d12video.h>

// Function to retrieve the reconstructed picture from an output sample.
// The output parameter will be set to a null pointer if the output sample
// does not have a reconstructed picture.
HRESULT RetrieveD3D12ReconstructedPictureFromOutputSample(
    _In_ IMFSample* outputSample,
    _COM_Outptr_result_maybenull_ ID3D12Resource** reconstructedPictureBuffer)
{
    RETURN_HR_IF_NULL(E_POINTER, reconstructedPictureBuffer);

    *reconstructedPictureBuffer = nullptr;

    RETURN_HR_IF_NULL(E_INVALIDARG, outputSample);

    // Retrieve the sample attributes
    wil::com_ptr_nothrow<IMFAttributes> sampleAttributes;
    RETURN_IF_FAILED(wil::com_query_to_nothrow(outputSample, &sampleAttributes));

    // Retrieve the IMFMediaBuffer associated with the reconstructed picture
    wil::com_ptr_nothrow<IMFMediaBuffer> mediaBuffer;
    HRESULT hr = sampleAttributes->GetUnknown(
        MFSampleExtension_VideoEncodeD3D12ReconstructedPicture,
        IID_PPV_ARGS(&mediaBuffer));
    RETURN_HR_IF(hr, hr != S_OK && hr != MF_E_ATTRIBUTENOTFOUND);

    if (mediaBuffer != nullptr)
    {
        wil::com_ptr_nothrow<IMFDXGIBuffer> dxgiBuffer;
        RETURN_IF_FAILED(mediaBuffer.query_to(&dxgiBuffer));

        wil::com_ptr_nothrow<ID3D12Resource> d3d12ReconBuffer;
        RETURN_IF_FAILED(dxgiBuffer->GetResource(IID_PPV_ARGS(&d3d12ReconBuffer)));

        // To use the buffer, use IMFD3D12SynchronizationObjectCommands to synchronize
        // (e.g., by invoking EnqueueResourceReadyWait on the consumer queue.)
        // To access the buffer on the CPU, use QueryInterface to access mediaBuffer as an
        // IMFMediaBuffer, and invoke the IMFMediaBuffer::Lock method.
        *reconstructedPictureBuffer = d3d12ReconBuffer.detach();
    }
    return S_OK;
}

Requirements

Requirement Value
Minimum supported client Windows 11, build 26100
Minimum supported server Windows Server 2025
Header Mfapi.h

See also

  • Last updated on