Uncompressed Video Buffers
This article describes how to work with media buffers that contain uncompressed video frames. In order of preference, the following options are available. Not every media buffer supports each option.
- Use the underlying Direct3D surface. (Applies only to video frames stored in Direct3D surfaces.)
- Use the IMF2DBuffer interface.
- Use the IMFMediaBuffer interface.
Use the Underlying Direct3D Surface
The video frame might be stored inside a Direct3D surface. If so, you can get a pointer to the surface by calling IMFGetService::GetService or MFGetService on the media buffer object. Use the service identifier MR_BUFFER_SERVICE. This approach is recommended when the component accessing the video frame is designed to use Direct3D. For example, a video decoder that supports DirectX Video Acceleration should use this approach.
The following code shows how to get the IDirect3DSurface9 pointer from a media buffer.
IDirect3DSurface9 *pSurface = NULL;
hr = MFGetService(
pBuffer,
MR_BUFFER_SERVICE,
__uuidof(IDirect3DSurface9),
(void**)&pSurface
);
if (SUCCEEDED(hr))
{
// Call IDirect3DSurface9 methods.
}
The following objects support the MR_BUFFER_SERVICE service:
Use the IMF2DBuffer Interface
If the video frame is not stored inside a Direct3D surface, or the component is not designed to use Direct3D, the next recommended way to access the video frame is to query the buffer for the IMF2DBuffer interface. This interface is designed specifically for image data. To get a pointer to this interface, call QueryInterface on the media buffer. Not all media buffer objects expose this interface. But if a media buffer does expose the IMF2DBuffer interface, you should use that interface to access the data, if possible, rather than using IMFMediaBuffer. You can still use the IMFMediaBuffer interface, but it might be less efficient.
- Call QueryInterface on the media buffer to get the IMF2DBuffer interface.
- Call IMF2DBuffer::Lock2D to access the memory for the buffer. This method returns a pointer to the first byte of the top row of pixels. It also returns the image stride, which is the number of bytes from the start of a row of pixels to the start of the next row. The buffer might contain padding bytes after each row of pixels, so the stride might be wider than the image width in bytes. Stride can also be negative if the image is oriented bottom-up in memory. For more information, see Image Stride.
- Keep the buffer locked only while you need to access the memory. Unlock the buffer by calling IMF2DBuffer::Unlock2D.
Not every media buffer implements the IMF2DBuffer interface. If the media buffer does not implement this interface (that is, the buffer object returns E_NOINTERFACE in step 1), you must use the IMFMediaBuffer interface interface, described next.
Use the IMFMediaBuffer Interface
If a media buffer does not expose the IMF2DBuffer interface, use the IMFMediaBuffer interface. The general semantics of this interface are described in the topic Working with Media Buffers.
- Call QueryInterface on the media buffer to get the IMFMediaBuffer interface.
- Call IMFMediaBuffer::Lock to access the buffer memory. This method returns a pointer to the buffer memory. When the Lock method is used, the stride is always the minimum stride for the video format in question, with no extra padding bytes.
- Keep the buffer locked only while you need to access the memory. Unlock the buffer by calling IMFMediaBuffer::Unlock.
You should always avoid calling IMFMediaBuffer::Lock if the buffer exposes IMF2DBuffer, because the Lock method can force the buffer object to the video frame into a contiguous memory block and then back again. On the other hand, if the buffer does not support IMF2DBuffer, then IMFMediaBuffer::Lock will probably not result in a copy.
Calculate the minimum stride from the media type as follows:
- The minimum stride might be stored in the MF_MT_DEFAULT_STRIDE attribute.
- If the MF_MT_DEFAULT_STRIDE attribute is not set, call the MFGetStrideForBitmapInfoHeader function to calculate the stride for most common video formats.
- If the MFGetStrideForBitmapInfoHeader function does not recognize the format, you must calculate the stride based on the definition of the format. In that case, there is no general rule; you need to know the details of the format definition.
The following code shows how to get the default stride for the most common video formats.
HRESULT GetDefaultStride(IMFMediaType *pType, LONG *plStride)
{
LONG lStride = 0;
// Try to get the default stride from the media type.
HRESULT hr = pType->GetUINT32(MF_MT_DEFAULT_STRIDE, (UINT32*)&lStride);
if (FAILED(hr))
{
// Attribute not set. Try to calculate the default stride.
GUID subtype = GUID_NULL;
UINT32 width = 0;
UINT32 height = 0;
// Get the subtype and the image size.
hr = pType->GetGUID(MF_MT_SUBTYPE, &subtype);
if (FAILED(hr))
{
goto done;
}
hr = MFGetAttributeSize(pType, MF_MT_FRAME_SIZE, &width, &height);
if (FAILED(hr))
{
goto done;
}
hr = MFGetStrideForBitmapInfoHeader(subtype.Data1, width, &lStride);
if (FAILED(hr))
{
goto done;
}
// Set the attribute for later reference.
(void)pType->SetUINT32(MF_MT_DEFAULT_STRIDE, UINT32(lStride));
}
if (SUCCEEDED(hr))
{
*plStride = lStride;
}
done:
return hr;
}
Depending on your application, you might know in advance whether a given media buffer supports IMF2DBuffer (for example, if you created the buffer). Otherwise, you must be prepared to use either of the two buffer interfaces.
The following example shows a helper class that hides some of these details. This class has a LockBuffer method that calls either Lock2D or Lock and returns a pointer to the start of the first row of pixels. (This behavior matches the Lock2D method.) The LockBuffer method takes the default stride as an input parameter and returns the actual stride as an output parameter.
class CBufferLock
{
public:
CBufferLock(IMFMediaBuffer *pBuffer) : m_p2DBuffer(NULL), m_bLocked(FALSE)
{
m_pBuffer = pBuffer;
m_pBuffer->AddRef();
m_pBuffer->QueryInterface(IID_IMF2DBuffer, (void**)&m_p2DBuffer);
}
~CBufferLock()
{
UnlockBuffer();
SafeRelease(&m_pBuffer);
SafeRelease(&m_p2DBuffer);
}
HRESULT LockBuffer(
LONG lDefaultStride, // Minimum stride (with no padding).
DWORD dwHeightInPixels, // Height of the image, in pixels.
BYTE **ppbScanLine0, // Receives a pointer to the start of scan line 0.
LONG *plStride // Receives the actual stride.
)
{
HRESULT hr = S_OK;
// Use the 2-D version if available.
if (m_p2DBuffer)
{
hr = m_p2DBuffer->Lock2D(ppbScanLine0, plStride);
}
else
{
// Use non-2D version.
BYTE *pData = NULL;
hr = m_pBuffer->Lock(&pData, NULL, NULL);
if (SUCCEEDED(hr))
{
*plStride = lDefaultStride;
if (lDefaultStride < 0)
{
// Bottom-up orientation. Return a pointer to the start of the
// last row *in memory* which is the top row of the image.
*ppbScanLine0 = pData + abs(lDefaultStride) * (dwHeightInPixels - 1);
}
else
{
// Top-down orientation. Return a pointer to the start of the
// buffer.
*ppbScanLine0 = pData;
}
}
}
m_bLocked = (SUCCEEDED(hr));
return hr;
}
void UnlockBuffer()
{
if (m_bLocked)
{
if (m_p2DBuffer)
{
(void)m_p2DBuffer->Unlock2D();
}
else
{
(void)m_pBuffer->Unlock();
}
m_bLocked = FALSE;
}
}
private:
IMFMediaBuffer *m_pBuffer;
IMF2DBuffer *m_p2DBuffer;
BOOL m_bLocked;
};
Related topics