Media Foundation Wrong Size for Video

Question

CComPtr<IMFSourceReader> r;  
CComPtr<IMFMediaType> NativeMT;  
MFCreateSourceReaderFromURL(L"file.mp4" 0, &r);  
r->GetCurrentMediaType((DWORD)MF_SOURCE_READER_FIRST_VIDEO_STREAM, &NativeMT);  
UINT wi = 0,he = 0;  
MFGetAttributeSize(NativeMT, MF_MT_FRAME_SIZE, &wi,&he)  

For some H265 videos taken with my phone (example: https://drive.google.com/file/d/12oH9x7MCW7YFZu1MDKGbOnIt4VvxrgtZ/view?usp=share_link) this returns wrong size.

Original video: 3840x2160 pixels
Returned: 3840x2172 pixels

I will follow up with more because videos like that fail to be converted with media foundation.

Answer 1

Hi... Thanks for reaching out. As the previous messages have indicated, Media Foundation does support H.265, although in some cases it may require the HEVC Video Extensions purchased via the Microsoft Store.

In looking through this issue, it does appear that Media Foundation is doing the proper thing. The HEVC encoder requires that the frame size be of certain size, and depending on other profile settings could be upwards of 32x32 pixel blocks (as is likely the case in this file). Since 2160 is not a multiple of 32, the encoder is encoding a larger frame size (in this case 2176) to satisfy that constraint. When the file is written, the header is usually then also marked with an indication of how many of these pixels are actually valid.

In the case of what Media Foundation is doing then, when parsing the file, the full frame size (3840x2176) is stored in the attribute retrieved by MF_MT_FRAME_SIZE, and the valid pixels are stored in MF_MT_MINIMUM_DISPLAY_APERTURE attribute, which you could use while building your application.

    MFVideoArea mfArea;
    ZeroMemory(&mfArea, sizeof(mfArea));
    hr = nativeMT->GetBlob(MF_MT_MINIMUM_DISPLAY_APERTURE, (UINT8*)&mfArea, sizeof(MFVideoArea), NULL);

The effect of this is applications, such as Windows Media Player Legacy, that don't understand these attributes, will display the fully encoded frame (including the invalid pixels), whereas newer applications that do understand it will properly display the expected frame size (3840x2160)

If you are still seeing additional errors transcoding content after accounting for the above, please provide additional details as to the specific error(s) that you are still seeing.

Answer 2

Hello MichaelChourdakis

The issue might be produced because Media Foundation doesn not support H265 codec:

https://learn.microsoft.com/en-us/windows/win32/medfound/supported-media-formats-in-media-foundation#video-codecs

--If the reply is helpful, please Upvote and Accept as answer--