Edit

Share via

Hardware Offloaded APO Effects

  • Article

In Windows 10, version 1511 and later, offloading of audio processing objects (APOs) is supported. In addition to possible performance enhancements, there are significant possible power savings available when using hardware offloaded APOs.

Two types of APOs can be loaded during hardware offload playback.

  1. Offload Stream Effects (OSFX)
  2. Offload Mode Effects (OMFX)

Hardware Offloaded APO Effects Overview

Hardware Offloaded Audio Processing and Hardware Offloaded APOs

In Windows 8, the audio engine has been redesigned to work with audio streams that have been offloaded to a hardware device that is separate from, but connected to, the computer's main audio system. This is referred to as hardware offloading. For more information, see Hardware-Offloaded Audio Processing.

The hardware offloading feature is primarily targeted for low power scenarios with larger buffer sizes. For example, during Low Power Audio (LPA) playback in the capable systems, the audio buffer size or periodicity may be set to 1 second so that the CPU doesn’t wake up frequently to process small buffers (e.g., at every 10 milliseconds).

Implementing hardware offloaded APOs along with hardware offloaded audio processing provides the ability to maximize power efficiency.

The following diagram shows the audio processing objects architecture. The right side of the diagram shows an application communicating down to hardware offloaded OSFX and OMFX effects.

Diagram showing audio driver architecture with application communicating to hardware offloaded OSFX and OMFX effects, drivers, and audio hardware.

Implementing Hardware Offloaded APO Effects

A hardware offloaded APO must follow the same basic requirements and design principles described in Audio Processing Object Architecture and Implementing Audio Processing Objects.

Supported Audio Format Implementation Guidelines

For hardware offloaded APOs, some additional consideration must be given to supported audio formats.

Each APO implements IAudioProcessingObject::IsInputFormatSupported method which is used during audio graph building to determine the output audio format and whether any format conversion is needed.

HRESULT IsInputFormatSupported(
  [in, optional]  IAudioMediaType *pOppositeFormat,
  [in, optional]  IAudioMediaType *pRequestedInputFormat,
  [out, optional] IAudioMediaType **ppSupportedInputFormat
);

Offload render endpoint can support a variety of formats including the default format supported by the host/system pin rendering. An Offload APO should support all of these formats so that rendering streams (with the supported formats) don’t have to go through any additional format conversion.

An offload SFX can implement format conversions and accept a broader range of formats. For example, if the Offload SFX provides headphone virtualizations (i.e., convert 5.1 channel audio to stereo), then it should return S_OK for the appropriate input/output pair in this method.

An offload SFX should review the offload pin supported formats and support/extend the capabilities together.

Offload MFX cannot change the format of the input stream, but it still needs to support the variety of formats offered by the offload endpoint and eliminate any unnecessary format conversion.

During rendering in the offload pin, only one stream is active on that pin and therefore there is no mixing of streams. So, processing the audio at both stream-level and mode-level is not necessary. Thus audio effects may not need to be enabled as both a stream effect and mode effect. Offloaded endpoints will support more streams, and depending on the processing architecture for a system, Offload processing may need to be factored into SFX/MFX.

INF file entries

Implement the following INF file entries to define the effects that will be loaded during offload playback. The INF file property key instructs the audio endpoint builder to set the CLSIDs for offloaded APOs into the effects property store. This information is used to build the audio graph that will be used to inform upper level apps what effects are in place.

Property Key GUID
PKEY_FX_Offload_StreamEffectClsid {D04E05A6-594B-4FB6-A80D-01AF5EED7D1D},11
PKEY_FX_Offload_ModeEffectClsid {D04E05A6-594B-4FB6-A80D-01AF5EED7D1D},12
PKEY_SFX_Offload_ProcessingModes_Supported_For_Streaming {D3993A3F-99C2-4402-B5EC-A92A0367664B},11
PKEY_MFX_Offload_ProcessingModes_Supported_For_Streaming {D3993A3F-99C2-4402-B5EC-A92A0367664B},12

Implementing Audio Processing Objects
Windows Audio Processing Objects