Using the Desktop Duplication API with HDR - interpreting DXGI_FORMAT_R16G16B16A16_FLOAT bytes

David Howell 20 Reputation points
2023-12-04T15:45:42.2066667+00:00

(note: I did not find a "DirectX" tag, please direct me to the right place if needed)

I am trying to use the Desktop Duplication API to capture an image from an HDR-enabled monitor. After some early troubles, I am able to get the desired pixel data from the D3D11_MAPPED_SUBRESOURCE structure. What is causing me some difficulty now is determining what exactly the format of the data that the (void*) 'pData' member is pointing to.

Here is a code snippet which illustrates the steps I am performing (details such as error checking omitted):

  1. Create factory object
CComPtr<IDXGIFactory6> pFactory;  
CreateDXGIFactory1( IID_IDXGIFactory6, reinterpret_cast<void**>( &pFactory ) );
  1. Gather adapter/output descriptors and interface instances
for ( UINT adapterIndex = 0u; true; ++adapterIndex )  
{  
   CComPtr<IDXGIAdapter1> pAdapter;
   pFactory->EnumAdapters1( adapterIndex, &pAdapter );  

   DXGI_ADAPTER_DESC3 adapterDescription3 = {};  
   pAdapter4->GetDesc3( &adapterDescription3 )  

   const CComQIPtr<IDXGIAdapter4> pAdapter4( pAdapter );  
   for ( UINT outputIndex = 0u; true; ++outputIndex )  
   {  
      CComPtr<IDXGIOutput> pOutput;
      pAdapter4->EnumOutputs( outputIndex, &pOutput );

      const CComQIPtr<IDXGIOutput6> pOutput6( pOutput );  
      DXGI_OUTPUT_DESC1 outputDescription = {};  
      pOutput6->GetDesc1( &outputDescription );

      if ( !outputDescription.AttachedToDesktop ) continue;  
   }  
}
  1. Create device interface instances
CComPtr<ID3D11Device> pDevice;  
CComPtr<ID3D11DeviceContext> pDeviceContext;  
D3D11CreateDevice( pAdapter, D3D_DRIVER_TYPE_UNKNOWN, ..., &pDevice, ..., &pDeviceContext );
  1. Call duplicate-output on device
constexpr DXGI_FORMAT formats[] =  
{  
    DXGI_FORMAT_R10G10B10A2_UNORM,  
    DXGI_FORMAT_R16G16B16A16_FLOAT,  
    DXGI_FORMAT_R8G8B8A8_UNORM,  
}  
CComPtr<IDXGIOutputDuplication> pDuplication;  
pOutput->DuplicateOutput1( pDevice, ..., formats, ..., pDuplication );  
  
DXGI_OUTDUPL_DESC outputDuplicationDescription;  
pDuplication->GetDesc( &outputDuplicationDescription );
  1. Copy the resource
DXGI_OUTDUPL_FRAME_INFO frameInfo;  
CComPtr<IDXGIResource> pDesktopResource;  
while ( true ) // keep going until a frame is retrieved  
{  
    pDuplication->AcquireNextFrame( timeout, &frameInfo, &pDesktopResource );  
    if ( frameInfo.LastPresentTime.QuadPart ) break;  
    pDuplication->ReleaseFrame();  
}  
  
const CComQIPtr<ID3D11Texture2D> pDesktopTexture( pDesktopResource );  
  
D3D11_TEXTURE2D_DESC desktopTextureDescription;  
pDesktopTexture->GetDesc( &desktopTextureDescription );  
  
D3D11_TEXTURE2D_DESC copyTextureDescription = { details omitted };  
  
CComPtr<ID3D11Texture2D> pCopyTexture;  
pDevice->CreateTexture2D( &copyTextureDescription, nullptr, &pCopyTexture );  
pDeviceContext->CopyResource( pCopyTexture, pDesktopTexture );
  1. Retrieve pixel data
D3D11_MAPPED_SUBRESOURCE pixelData = {};  
pDeviceContext->Map( pCopyTexture, 0u, D3D11_MAP_READ, 0u, &pixelData );  
// fyi, done later: pDeviceContext->Unmap( pCopyTexture, 0u );

What I have at this point is:

  • adapterDescription3.Description = "NVIDIA RTX 3500 Ada Generation Laptop GPU"
  • outputDescription.DeviceName = "\\.\DISPLAY2"
  • outputDescription.DesktopCoordinates = {LT(0, 0) RB(3840, 2160) [3840 x 2160]}
  • outputDescription.AttachedToDesktop = 1 (TRUE)
  • outputDescription.Monitor = 0x.... (HMONITOR)
  • outputDescription.BitsPerColor = 10
  • outputDescription.ColorSpace = DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020; // HDR display with all Advanced Color capabilities
  • outputDuplicationDescription.ModeDesc.Format = DXGI_FORMAT_R16G16B16A16_FLOAT
  • desktopTextureDescription.Format = DXGI_FORMAT_R16G16B16A16_FLOAT
  • pixelData.pData = 0x... (void*) // 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3C, 0x99, ...
  • pixelData.RowPitch = 30720 // width (3840) * 8
  • pixelData.DepthPitch = 66355200 // RowPitch * height (2160)

(my desktop is a test image which has pixels starting in the upper-left whose RGB values are:
0x000000 | 0xFFFFFF | 0xFF0000 | 0x123456 | 0xF0F0F0 | ... | 0xF0F0F0)

My question is, what is the format of the data (bytes) in pixelData.pData? I've seen it mentioned that it is "half-float" values in the range 0.0 to 1.0, which represent ratios of the 10-bit pixel values, but this doesn't jive with the pixelData.RowPitch, which shows that there are 8 bytes per color (64 bits).

What is the format of those bytes? I want to do some sort of HDR-to-SDR conversion on that data and I'm not sure exactly how to handle what's there. I'm not seeing any clues in any Microsoft sample in GitHub, nor in any other open source code in GitHub.

Windows API - Win32
Windows API - Win32
A core set of Windows application programming interfaces (APIs) for desktop and server applications. Previously known as Win32 API.
2,511 questions
{count} votes