Edit

Handling Stream Changes

  • Article

This topic describes how a Media Foundation transform (MFT) should handle format changes during streaming.

Important

This topic does not apply to encoders. Encoders should not propagate format changes as described in this topic. Encoders should only accept an input type that matches the currently configured output type.

 

Overview of Format Changes

Generally, there are two reasons that a format can change during streaming.

  • The client might switch to a stream with a new format. For example, in digital television, this can occur due to a channel change.
  • In some video formats, such as H.264, the bitstream can signal a format change. Such changes might include changes in field dominance, video resolution, or pixel aspect ratio.

If the encoding type changes, the client might need to remove the MFT from the pipeline and replace it with another MFT. (For example, the client might need to swap in a new decoder.) This topic does not cover that situation. This topic covers only the case where the current MFT can handle the new format.

If the format changes, the MFT might require a new input type, a new output type, or both.

  • Changes to input type are initiated by the client. An MFT never changes its own input type.
  • Changes to output type are initiated by the MFT. The MFT signals that it requires a new output type, and the client negotiates the new output type with the MFT.

Thus, three distinct cases are possible:

  • The client sets a new input type. The MFT consumes the new format, with no change to its output type.
  • The client sets a new input type, and this triggers a change in the output type.
  • The input type does not change, but the MFT detects a format change in the bitstream, which requires a new output type.

Implementing Format Changes

The remainder of this topic describes how the client should process a format change, and how to implement format changes in an MFT.

Output Type

Any MFT can initiate a change to its output type, as follows:

  1. The client calls IMFTransform::ProcessOutput. The MFT responds as follows:
    1. The MFT does not produce an output sample in ProcessOutput.
    2. The MFT sets the MFT_OUTPUT_DATA_BUFFER_FORMAT_CHANGE flag in the dwStatus member of the MFT_OUTPUT_DATA_BUFFER structure.
    3. The ProcessOutput method returns the error code MF_E_TRANSFORM_STREAM_CHANGE.
  2. The client calls IMFTransform::GetOutputAvailableType. This method returns an updated set of output types.
  3. The client calls SetOutputType to set a new output type.
  4. The client resumes calling ProcessInput/ProcessOutput.

Input Type

Changes to the input type are initiated by the client, never by the MFT. If the input type changes, it might trigger a change to the output type.

The exact sequence of events depends on the value of the MFT_SUPPORT_DYNAMIC_FORMAT_CHANGE attribute.

Value Description
FALSE Before the client sets a new input type, it must drain the MFT.
TRUE The client can set a new input type without draining the MFT.

 

An MFT exposes this attribute through its IMFTransform::GetAttributes method. The default value of this attribute is FALSE; if the MFT does not set the attribute, treat the value as FALSE.

MFT_SUPPORT_DYNAMIC_FORMAT_CHANGE is FALSE

  1. The client sends the MFT_MESSAGE_COMMAND_DRAIN message.
  2. The client drains the MFT by calling IMFTransform::ProcessOutput until ProcessOutput returns MF_E_TRANSFORM_NEED_MORE_INPUT.
  3. The client calls IMFTransform::SetInputType to set the new input type.
  4. The MFT validates the input type. If the type is invalid, SetInputType returns MF_E_INVALIDMEDIATYPE or another error code. Otherwise, SetInputType returns S_OK.
  5. Assuming the input type is valid, the MFT evaluates whether the output type also changes. If not, streaming continues, and no further action is required.
  6. If the output type changes:
    1. The MFT invalidates its current output media type, and updates the list of available output media types.
    2. The next call to ProcessOutput returns MF_E_TRANSFORM_STREAM_CHANGE, as described in the previous section.
    3. The client calls IMFTransform::GetOutputAvailableType to get the updated list of output types.
    4. The client calls SetOutputType.

MFT_SUPPORT_DYNAMIC_FORMAT_CHANGE is TRUE

  1. The client calls IMFTransform::SetInputType to set the new input type.
  2. The MFT validates the input type. If the type is invalid, SetInputType returns MF_E_INVALIDMEDIATYPE or another error code. Otherwise, SetInputType returns S_OK.
  3. Assuming the input type is valid, the MFT evaluates whether the output type also changes. If not, streaming continues, and no further action is required.
  4. Before the output type changes, the MFT must process any cached input samples, as follows:
    1. The MFT does not invalidate its current output type.
    2. The MFT produces as much output as it can from the cached input samples.
    3. It is optional whether the MFT accepts new input samples while it processes the cached samples. If so, the new input samples will use the new input format, so the MFT must keep track of the point when the format changed.
  5. After the MFT processes all of the samples that it received before the input type changed, the IMFTransform::ProcessOutput returns MF_E_TRANSFORM_STREAM_CHANGE.
  6. The MFT invalidates its current output type, and updates the list of available output media types.
  7. The client negotiates the new output type, as described previously.

Asynchronous MFTs must return the value TRUE for the MFT_SUPPORT_DYNAMIC_FORMAT_CHANGE attribute. When using an asynchronous MFT, the client can assume that the MFT_SUPPORT_DYNAMIC_FORMAT_CHANGE attribute is set to TRUE.

When MFT_SUPPORT_DYNAMIC_FORMAT_CHANGE is TRUE, the main difference is that the client is not required to drain the MFT before setting a new input type. As a result, the input type might change while the MFT is holding onto input samples. It is important that the MFT does not simply drop these samples. Also, the output type cannot change until the MFT processes all of its cached data.

The previous paragraph applies especially to video decoders, which can receive inter-coded frames out of temporal order, and thus need to cache them. If an MFT does not cache input samples, draining is essentially a no-op. In that case, the MFT can set MFT_SUPPORT_DYNAMIC_FORMAT_CHANGE to FALSE (or leave the attribute unset).

Also, note that every MFT is expected to handle format changes correctly after being drained. The MFT_SUPPORT_DYNAMIC_FORMAT_CHANGE attribute indicates whether the MFT supports format changes without draining.

Change in Interlace Mode

Changes in the video interlacing mode are a special case, because they do not invalidate the current media type. Instead, the interlace mode is specified for each video frame by setting attributes on the media sample. A video MFT should check each input sample for the presence of these flags.

The interlace mode can change when the field dominance switches from top-field to bottom-field, or when the video switches between progressive and interlaced pictures.

For more information, see Interlace Flags on Samples.

Writing a Custom MFT