Media Foundation not properly rendering in Direct2D (3D Compatible) IDXGIRenderTarget

John Jacko 21 Reputation points
2020-12-29T04:39:04.843+00:00

I'm trying to create an application that can render Direct2D Content, Direct3D Content, and can use Media Foundation to play videos. As of right now, I'm satisfied with how Direct2D and Direct3D are working together.

There are four major objects I defined to organize all of this:

  1. TWindow - object that manages the Window
  2. TWindowEngine - object that holds Direct3D resources
  3. DrawingBoard - Draws against a Bitmap-Render Target and a DXGI-RenderTarget
  4. TPlayer - my own version of the CPlayer in this tutorial

The Way it works is when I draw 2D or 3D content, I follow the following steps:

  1. Call the TWindowEngine to initialize a Direct3D Draw
  2. Call the Drawing Board to BeginDraw on the Bitmap Render Target
  3. Make various calls on that Bitmap Render Target
  4. Call End Draw on the DrawingBoard
    a. Call EndDraw on the Bitmap Render Target
    b. Call BeginDraw on the DXGIRenderTarget
    c. Retrieve the Bitmap from the BitmapRenderTarget and Draw it on the DXGI RenderTarget
    d. Call EndDraw on the DXGIRenderTarget
  5. Call the Swap Chain to Present.

As of right now, I set up Direct 3D before doing anything related to Media Foundation. When I do set up the TPlayer, it is similar to how the CPlayer in the tutorial is set up with a notable exception.

HRESULT TPlayer::CreateSession(TrecPointer<TWindowEngine> en)  
{  
 HRESULT hr = CloseSession();  
  
 if (FAILED(hr))  
 return hr;  
  
 if (en.Get() && en->getDeviceD())  
 {  
 IMFAttributes* atts;  
 MFCreateAttributes(&atts, 5);  
  
 UINT token = 0;  
 IMFDXGIDeviceManager* manager;  
 MFCreateDXGIDeviceManager(&token, &manager);  
  
  
 ID3D11Device* dev = en->getDeviceD();  
 ID3D10Multithread* multi = nullptr;  
 if (SUCCEEDED(dev->QueryInterface(__uuidof(ID3D10Multithread), (void**)&multi)))  
 {  
 multi->SetMultithreadProtected(TRUE);  
 multi->Release();  
 }  
  
 manager->ResetDevice(dev, token);  
 atts->SetUINT32(MF_SA_D3D11_AWARE, TRUE);  
 atts->SetUnknown(MF_SOURCE_READER_D3D_MANAGER, manager);  
 atts->SetUINT32(MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS, TRUE);  
 atts->SetUINT32(MF_SOURCE_READER_ENABLE_ADVANCED_VIDEO_PROCESSING, TRUE);  
 atts->SetUINT32(MF_SOURCE_READER_ENABLE_VIDEO_PROCESSING, TRUE);  
  
  
  
  
 hr = MFCreateMediaSession(atts, &m_pSession);  
  
                // Release Calls  
  
 }  
 else  
 hr = MFCreateMediaSession(nullptr, &m_pSession);  
 if (FAILED(hr))  
 return hr;  
  
 hr = m_pSession->BeginGetEvent(this, nullptr);  
 if (SUCCEEDED(hr))  
 m_state = PlayerState::ps_ready;  
 return hr;  
}  

I figured that since Media Foundation sets up Direct3D internally, it might be possible to feed it my Direct3D resources, though I suspect that when a Media Sink is created using the Window Handle, that handle was locked or invalidated when the Swap Chain was created.

Here is a small snippet of the Swap Chain description and the flags used to create the ID3D11 Device...

 DXGI_SWAP_CHAIN_DESC swapChainDescription;  
  
  
  
 // Initialize the swap cahin  
 swapChainDescription.BufferCount = 2;  
 swapChainDescription.BufferDesc.Width = Location.right - Location.left;  
 swapChainDescription.BufferDesc.Height = Location.bottom - Location.top;  
 swapChainDescription.BufferDesc.Format = DXGI_FORMAT_B8G8R8A8_UNORM;  
 swapChainDescription.BufferDesc.RefreshRate.Numerator = 30;  
 swapChainDescription.BufferDesc.RefreshRate.Denominator = 1;  
 swapChainDescription.BufferUsage = DXGI_USAGE_RENDER_TARGET_OUTPUT;  
 swapChainDescription.OutputWindow = window;  
 swapChainDescription.Windowed = true;  
 swapChainDescription.SampleDesc.Count = 1;  
 swapChainDescription.SampleDesc.Quality = 0;  
 swapChainDescription.BufferDesc.Scaling = DXGI_MODE_SCALING_UNSPECIFIED;  
 swapChainDescription.BufferDesc.ScanlineOrdering = DXGI_MODE_SCANLINE_ORDER_UNSPECIFIED;  
 swapChainDescription.Flags = 0;  
 swapChainDescription.SwapEffect = DXGI_SWAP_EFFECT_FLIP_DISCARD;// DXGI_SWAP_EFFECT_DISCARD;  
  
 unsigned int flags = D3D11_CREATE_DEVICE_BGRA_SUPPORT | D3D11_CREATE_DEVICE_VIDEO_SUPPORT;  
  
#if _DEBUG  
 flags |= D3D11_CREATE_DEVICE_DEBUG;  
#endif  
  
  
 HRESULT results;  
  
 ID3D11Device *dev;  
 ID3D11DeviceContext* devCon;  
 results = D3D11CreateDevice(0, D3D_DRIVER_TYPE_HARDWARE, 0, flags,  
 0, 0, D3D11_SDK_VERSION, &dev, &version, &devCon);  
  

Incidentally, If I don't set up 3D (and my DrawingBoard uses an HwndRenderTarget in place of a DXGIRenderTarget), then I can get my app to flicker between Direct2D and Media Foundation showing up on my window.

Is there a better way to get Media Foundation to work with my Direct2D and Direct3D resources? Are there a couple tweaks I could make to get my current set-up to work?

Note: I am able to hear the video, so Media Foundation is running. It's just that it doesn't show when I set up Direct3D and it Conflicts with Direct2D when [my] 3D is out of the picture.

Windows API - Win32
Windows API - Win32
A core set of Windows application programming interfaces (APIs) for desktop and server applications. Previously known as Win32 API.
2,416 questions
C++
C++
A high-level, general-purpose programming language, created as an extension of the C programming language, that has object-oriented, generic, and functional features in addition to facilities for low-level memory manipulation.
3,525 questions
{count} votes