Video Interlacing
This topic describes how media sources and decoders should handle interlaced video content.
To decode and render interlaced video correctly, the following information is needed:
Progressive or interlaced. A video stream can contain progressive frames, interlaced frames, or a mix of both.
Field dominance. Field dominance describes which field appears first, the upper field or the lower field.
Repeat first field. This flag is used in 3:2 pulldown, when the frame is progressive but the stream is interlaced. In this context, the first field can be the upper or lower field.
Interleaved fields or single field. A sample can hold either a single field, or two interleaved fields. If a sample contains a single field, the sample height is half the frame height, because the sample contains only half of the scan lines for a frame. Interleaved fields are recommended unless the characteristics of the source content dictate otherwise.
Any of these characteristics can change from one sample to the next. However, video components need to know something about the overall content before streaming begins. For example, if the video is interlaced, the enhanced video renderer (EVR) needs to reserve video memory for the deinterlacing. If the video is entirely progressive frames, on the other hand, the EVR can optimize the rendering pipeline. Adding a deinterlacing step to the pipeline increases the rendering latency.
Information about interlacing is stored in two places:
General information about the interlacing in a stream is placed in the media type. For more information about media types, see Media Types.
Information that can change with each sample is placed on the sample as an attribute. For more information about samples, see Media Samples.
Interlace Information in the Media Type
The MF_MT_INTERLACE_MODE attribute on the media type describes how the stream as a whole is interlaced. The value of this attribute is a member of the MFVideoInterlaceMode enumeration. A video media type should always have this attribute.
- If the stream contains only progressive frames, with no interlaced frames, use MFVideoInterlace_Progressive.
- If the stream contains only interlaced frames, and every sample contains two interleaved fields, use MFVideoInterlace_FieldInterleavedUpperFirst or MFVideoInterlace_FieldInterleavedLowerFirst.
- If the stream contains only interlaced frames, and every sample contains a single field, use MFVideoInterlace_FieldSingleUpper or MFVideoInterlace_FieldSingleLower. If the fields alternate between upper and lower, then it does not matter which of these two values is used. If the format contains just upper fields, or just lower fields, then set the value that corresponds to the content.
- If the stream contains a mix of interlaced and progressive frames, or if the field dominance switches, set the media type to MFVideoInterlace_MixedInterlaceOrProgressive. Use sample attributes to describe each frame.
The following table summarizes this attribute.
| MF_MT_INTERLACE_MODE | Interlaced? | Samples | First field |
|---|---|---|---|
| MFVideoInterlace_Progressive | No | Progressive frame | Not applicable |
| MFVideoInterlace_FieldInterleavedUpperFirst | Yes | Interleaved fields | Upper first |
| MFVideoInterlace_FieldInterleavedLowerFirst | Yes | Interleaved fields | Lower first |
| MFVideoInterlace_FieldSingleUpper | Yes | Single field | Upper first |
| MFVideoInterlace_FieldSingleLower | Yes | Single field | Lower first |
| MFVideoInterlace_MixedInterlaceOrProgressive | Can vary | Interleaved fields or progressive frames | Can vary |
Interleaved fields and single fields cannot be mixed. Switching from one to another requires a media type change.
Interlace Flags on Samples
Information that can change from one sample to the next is indicated using sample attributes. Use the IMFSample interface to get or set these attributes.
All of the interlacing attributes listed in this section have Boolean values. Effectively, each of these attributes can have three values: either TRUE, FALSE, or not set. If an attribute is not set, the value is taken from the media type. If an attribute is set, the value overrides the media type. Some combinations of flags and media types are not valid.
| Attribute | Description |
|---|---|
| MFSampleExtension_Interlaced | If TRUE, the frame is interlaced. If FALSE, the frame is progressive. Set this attribute on every sample if the media type is MFVideoInterlace_MixedInterlaceOrProgressive. |
| MFSampleExtension_BottomFieldFirst | The meaning of this flag depends on whether the samples contain interleaved fields or single fields.
|
| MFSampleExtension_RepeatFirstField | If TRUE, the first field is repeated. If FALSE or not set, the first field is not repeated. |
| MFSampleExtension_SingleField | If TRUE, the sample contains a single field. If FALSE, the sample contains interleaved fields. |
The following table shows which flags are required, optional, or prohibited, based on the media type.
| Media Type | Interlaced Flag | BottomFieldFirst Flag | RepeatFirstField Flag | SingleField Flag |
|---|---|---|---|---|
| Progressive | Optional; if set, must be FALSE. | Do not set. | Do not set. | Do not set. |
| Interleaved fields | Optional; if set, must be TRUE. | Optional; if set, must match media type. | Do not set. | Optional; if set, must be FALSE. |
| Single fields | Optional; if set, must be TRUE. | Required. | Do not set. | Set to TRUE. |
| Mixed | Required. | Required. | Required. | Optional; if set, must be FALSE. |
In the cases where the attribute is optional, the media type already defines the information. It is valid to set the attribute to match, but not required.
For example, if the media type is MFVideoInterlace_Progressive, it implies that all frames in the stream are progressive. Therefore, you can either set the MFSampleExtension_Interlaced attribute to FALSE, or leave the attribute unset.
Recommendations
This section contains recommendations for various types of content.
- The video is all progressive frames.
Set the media type to MFVideoInterlace_Progressive.
Do not set the MFSampleExtension_Interlaced attribute, or set it to FALSE on every frame.
Do not set the MFSampleExtension_BottomFieldFirst, MFSampleExtension_RepeatFirstField, or MFSampleExtension_SingleField attributes.
- The video is all interlaced fields with the same field dominance. Samples contain interleaved fields.
Set the media type to MFVideoInterlace_FieldInterleavedUpperFirst or MFVideoInterlace_FieldInterleavedLowerFirst.
Do not set the MFSampleExtension_Interlaced attribute, or set it to TRUE on every frame.
Do not set the MFSampleExtension_BottomFieldFirst attribute, or set the value on every frame to match the media type.
Do not set the MFSampleExtension_RepeatFirstField attribute, or set it to FALSE on every frame.
Do not set the MFSampleExtension_SingleField attribute, or set it to FALSE on every frame.
- The video contains a mix of interlaced and progressive frames, with repeated fields and varying field dominance (for example, DVD video).
Set the media type to MFVideoInterlace_MixedInterlaceOrProgressive.
On every frame, set the MFSampleExtension_Interlaced, MFSampleExtension_BottomFieldFirst, and MFSampleExtension_RepeatFirstField attributes.
Do not set the MFSampleExtension_SingleField attribute, or set it to FALSE on every frame.
- The video is interlaced and samples contain single fields.
Set the media type to MFVideoInterlace_FieldSingleUpper or MFVideoInterlace_FieldSingleLower.
On every frame, set the MFSampleExtension_BottomFieldFirst attribute.
Do not set the MFSampleExtension_Interlaced attribute, or set it to TRUE on every frame.
Do not set the MFSampleExtension_RepeatFirstField attribute, or set it to FALSE on every frame.
Do not set the MFSampleExtension_SingleField attribute, or set it to TRUE on every frame.
Most video content falls into one of these categories.
MPEG-2 Mappings
For MPEG-2 content, use the following mappings to convert the MPEG-2 flags to Media Foundation sample attributes.
picture_structure
| Value | Sample Attribute |
|---|---|
| frame | MFSampleExtension_SingleField = FALSE |
| top_field | MFSampleExtension_SingleField = TRUE MFSampleExtension_BottomFieldFirst = FALSE |
| bottom_field | MFSampleExtension_SingleField = TRUE MFSampleExtension_BottomFieldFirst = TRUE |
progressive_frame
| Value | Sample Attribute |
|---|---|
| 0 | MFSampleExtension_Interlaced = TRUE |
| 1 | MFSampleExtension_Interlaced = FALSE |
top_field_first
| Value | Sample Attribute |
|---|---|
| 0 | MFSampleExtension_BottomFieldFirst = TRUE |
| 1 | MFSampleExtension_BottomFieldFirst = FALSE |
repeat_first_field
| Value | Sample Attribute |
|---|---|
| 0 | MFSampleExtension_RepeatFirstField = FALSE |
| 1 | MFSampleExtension_RepeatFirstField = TRUE |
Single-Field Samples
If the media type is MFVideoInterlace_FieldSingleUpper or MFVideoInterlace_FieldSingleLower, it means that each sample contains a single field. However, the media type describes the entire frame. Therefore, each buffer contains only half the number of field lines given in the media type. For example, if the media type describes the video as 720 × 480, each field contains 240 scan lines, and therefore each buffer contains only 240 rows of pixels. If you write a component that accepts media types with single-field samples, you must take this fact into account when you access the data in the buffer.
The same rule applies to the geometric aperture (MF_MT_GEOMETRIC_APERTURE attribute) and minimum display aperture (MF_MT_MINIMUM_DISPLAY_APERTURE attribute). These regions are specified in terms of the entire frame, not the individual fields.
DirectShow Mappings
In DirectShow, per-sample interlacing information is contained in the dwTypeSpecificFlags member of the AM_SAMPLE2_PROPERTIES structure. The following table shows the equivalent attributes for Media Foundation.
| DirectShow sample flag | Media Foundation sample attribute |
|---|---|
| AM_VIDEO_FLAG_INTERLEAVED_FRAME | MFSampleExtension_SingleField = FALSE. |
| AM_VIDEO_FLAG_FIELD1 | MFSampleExtension_Interlaced = TRUE. MFSampleExtension_SingleField = TRUE. MFSampleExtension_BottomFieldFirst = FALSE. |
| AM_VIDEO_FLAG_FIELD2 | MFSampleExtension_Interlaced = TRUE. MFSampleExtension_SingleField = TRUE. MFSampleExtension_BottomFieldFirst = TRUE. |
| AM_VIDEO_FLAG_WEAVE | MFSampleExtension_Interlaced = FALSE. (This flag indicates that the driver should not deinterlace the two fields.) |
| AM_VIDEO_FLAG_FIELD1FIRST | MFSampleExtension_BottomFieldFirst = FALSE. If the content is interlaced and the AM_VIDEO_FLAG_FIELD1FIRST flag is not present, set this attribute to TRUE. |
| AM_VIDEO_FLAG_REPEAT_FIELD | MFSampleExtension_RepeatFirstField = TRUE. If the AM_VIDEO_FLAG_REPEAT_FIELD flag is not present, set this attribute to FALSE. |
If the DirectShow sample does not contain sample flags, use the value of dwInterlaceFlags from the VIDEOINFOHEADER2 structure:
| DirectShow interlace flag | Media Foundation sample attribute |
|---|---|
| AMINTERLACE_IsInterlaced | MFSampleExtension_Interlaced = TRUE. |
| AMINTERLACE_1FieldPerSample | MFSampleExtension_SingleField = TRUE. |
| AMINTERLACE_Field1First | MFSampleExtension_BottomFieldFirst = FALSE. |
Related topics