This document is to complement the original Expression Encoder 4 SP1 GPU Encoding white paper and discuss the changes that SP2 introduced on this subject. The latest service pack includes a major revision of the CUDA H.264 encoder as well as the introduction of Intel Sandy Bridge QSV support, which both will be discussed in depth below.

Please note that an updated performance report for Cuda GPU Encoding as well as a new one for QSV GPU Encoding are available.

General GPU Encoding Recommendations

To Maximize Performance

As before, we recommend applying the proper GPU for video processing load. We provide below basic guidelines for optimizing your performance using a GPU to encode. Using proper GPU hardware and a conservative "GPU Stream" setting can reduce the chances of maxing out the GPU resources. Another way to reduce the GPU memory footprint is by lowering the "Number of Reference Frames". GPU-Z is a great freeware to monitor the load and memory usage.

To calculate the appropriate number of GPU streams setting for a specific scenario, we also created a simple performance tool. More information is available here.

Finally, since available GPU memory is more important than ever, we recommend closing any other applications to minimize GPU memory usage outside of Encoder.

To Maximize Quality

• Consider using 1-pass constrained VBR over CBR.
• Make the "Buffer Window" longer. Note that this will impact latency.
• Use accurate pre-processing settings (aka de-interlacing and resizing methods) for a given source. For example, using the wrong de-interlacing method can introduce weave artifacts, which are hard to encode and will significantly impact overall output quality.
• Consider removing any letterboxing introduced by the source and/or the encode settings. Those introduce hard high-contrast edges which are hard to encode and will also impact output quality.

Second Generation of CUDA GPU Encoding

In collaboration with our partners at Main Concept, a lot of work was done on the CUDA H.264 encoder to optimize the performance of the GPU, as well as improving output quality and adding new encode functionality like true CBR and HRD support. Those changes have significantly impacted its behavior, especially in terms of performance:

1. GPU memory requirements have doubled using with the same settings when compared to SP1.
2. GPU load has more than doubled too.
3. CPU load has been reduced when encoding with CUDA.

In other words, you will experience better quality and less CPU load, at the cost of higher GPU load and some overall performance loss.

Finally, because the CUDA encoder doesn't support 2-pass encodings, they have been disabled. Please note that while Encoder will revert to software encoding if a 2-pass encode is selected, the new 1-pass H.264 VBR encoding options are fully supported in both CUDA and QSV-based encoding.

Recommendations

Because of the new behavior, we now recommend using only higher-end CUDA GPUs for this feature. A rough suggestion would be to get CUDA hardware with a minimum of 200 CUDA cores per HD stream to be encoded via CUDA and at the least 1 GB of video memory (2 GB or more preferred). It means that for encoding a single HD stream on the GPU, at the least a GeForce GTX 550 Ti or Quadro 2000 GPU is recommended, while encoding 2 or more HD streams on the GPU would require at the least a GeForce GTX 570, Quadro 6000 or Tesla C2050 card. As in Encoder SP1, multiple video cards can be used to speed up Smooth Streaming encodes. Note that we also changed our default of number of GPU streams to 2, which can be tweaked to take full advantage of the hardware used.

Since the GPU is taking more of the load, matching the CPU may not be as important for single stream encodes. Because most current CUDA GPUs can't take more that 2-3 HD streams at a time, a powerful CPU is still required for HD Smooth Streaming encodes.

Finally, make sure to install the latest Nvidia drivers available from Nvidia's website here. Note that the GPU driver version is required to be higher than 267.91 for SP2.

New Intel Sandy Bridge QSV GPU Encoding Support

What is QSV?

Intel Quick Sync Video (QSV) is a GPU acceleration technology available on some Intel processors that can dramatically increase the speed of transcoding media files. It is available on most 2^nd generation i3, i5 and i7 processors (aka "Sandy Bridge"), as well as some newer Xeon CPUs, like the E3-1285. More information about QSV is available on Intel's website. Please note that the newly Sandy Bridge E CPU line (example: i7-3960X) does not have an embedded GPU, and thus does not support QSV.

QSV encoding support has been integrated in Encoder 4 Pro SP2, which can be used in a similar fashion as CUDA encoding to speed up MP4 and H.264-based Smooth Streaming encode operations.

Usage

QSV GPU encoding works the same way than CUDA encoding as explained in the Encoder SP1 GPU Encoding white paper. Because the two GPU-based H.264 encoders behave very differently, Expression Encoder 4 Pro SP2 provides a way to control the number of streams independently between CUDA and QSV, which are defaulting to 2 and 8 respectively (see below). As with CUDA, QSV encoding is turned on by default in the Encoder application and turned off in the SDK.

As for the SDK, the QSV device is enumerated in the H264EncodeDevices.Devices along with the CUDA devices. A new H264EncodeDevice property, DeviceType of type H264DeviceType, provides a way to differentiate between the two GPU types. H264EncodeDevices.MaxNumberGPUStream keeps controlling the number of streams to be encoded by CUDA, while H264EncodeDevices.MaxNumberIntelHDGraphicsStream controls the streams encoded by QSV.

Multi GPU Encoding

Both CUDA and QSV encoding can be used together, either on one multi-stream encode or multiple instances of encoder, which may provide even better performance results. While encoding multiple streams on a PC with both technologies, the streams will be allocated in a snake pattern starting with the Intel GPU (1-2-2-1, 1-2-3-3-2-1) from the highest resolution stream down.

It's worth mentioning that, when used with a discrete GPU, Intel QSV requires the Intel GPU to be enabled as default video adapter in the BIOS and needs to be active and connected to a monitor. Note that LucidLogix Virtu technology, bundled with some Sandy Bridge PCs, enables virtualization of the discrete GPU through the Intel GPU, enabling the usage of both QSV and Cuda and using only one output. More information is available on LucidLogix' website.

Limitations

Like the CUDA implementation, the QSV supports only a subset of encode settings. For instance, those encoding features are not supported:

1. Reference B-frames
2. Adaptive B-frames
3. Multi pass encoding (2-pass encodes will use software encoding)

QSV encoding is not supported on XP.

An active Intel HD Graphics GPU is required to be accessible. This means that:

1. The PC's motherboard needs to support the Intel Accelerator GPU (most 1155-H67 and 1155-Z68 motherboards do, but all 1155-P67 don't).
2. The GPU needs to be connected to a screen and be activated.
3. Accessing the PC via Remote Access, VM, or via a service isn't supported.
4. If the PC is locked or accessed via Remote Desktop, the next encode operation will fail to use the GPU since it will not be available.

As with CUDA, QSV encoding is an approximation of what the software-only based encoding can provide. Output quality differences, especially at low bitrates should be expected. Because of this, we recommend using the software H.264 encoder if highest quality results are a priority over higher performance.

Recommendations

The integrated Intel Accelerator 3000 GPU, found in high-end mobile CPUs and in the K line of 2^nd generation i5 and i7 based desktops as well as on some Xeon and i3 CPUs, is highly recommended. See the Intel website for more details.

Because memory is shared, we highly recommend having a minimum of 4 GB for single stream encoding and 8 GB or more for Smooth Streaming and/or parallel encoding. Running out of system memory will greatly affect performance.

It is highly recommended to install the very latest drivers directly from the Intel website to ensure best stability and reliability.

While the QSV encoding can take a large load of concurrent streams, it is worth noting that a blend of hardware and software encoding will likely result in better performance in most Smooth Streaming cases.

Known Issues

A few issues were found in the recent sets of Intel drivers currently available. While all of them have been investigated and resolved, some of the fixes didn't make it into the latest drivers, but will be available in the next driver version (ETA early 2012). Here is the short list:

1. When using either Intel's driver 8.15.10.2361 or 8.15.10.2509, about 5% of H.264 Smooth Streaming encodes using QSV will hang at the start of the encode or may contain a corrupted stream. This issue was fixed in the driver 8.15.10.2559.
2. When using Intel's driver 8.15.10.2361 and encoding to QVBR in H.264 Main or High profile using QSV, output will be corrupted. The issue was fixed in the driver 8.15.10.2509.
3. Windows 8 Beta Users: Intel's current Windows 8 drivers do not yet support QSV, but will be in the future.

Other GPU Acceleration Technologies

OpenCL and DirectCompute technologies were evaluated but deemed not ready for integration in Expression Encoder 4 Pro SP2. We are planning to continue monitoring them and investigating potential integration in the future.

General Encoding

Support for Intel QSV GPU encoding (Pro only) – GPU-accelerated encoding has added support for the Intel HD Graphics GPU included in Sandy Bridge processors, in addition to the improved Nvidia CUDA support. You can use both of these devices when encoding for increased speed and performance. More information available here. QSV and Cuda performance reports are also available.

Support for 1 and 2-pass VBR for offline H.264 encodes (Pro only) – You can now select 1-pass VBR for MP4 encodes as well as both 1-pass and 2-pass VBR for H.264-based Smooth Streaming encodes.

Support for more AAC audio options (Pro only) – You can now choose multiple-channel audio along with more bitrate and sample rate options when you use the AAC codec.

Support for SRS audio encoding – You can now choose to enable SRS to convert an audio stream from multiple audio channels to two SRS 5.1 surround sound channels.

Live

Manage publishing points from within Expression Encoder (Pro only) – In a Live Broadcasting Project, you can now create, restart or update publishing points on an IIS Smooth Streaming server. You can also choose to automatically update a broadcast’s EventID.

Live crop – You can now crop your live or file sources in a Live Broadcasting project.

Support for Windows Media Streaming MBR in a Live broadcast – You can now add multiple bitrate streams to your WM Streaming broadcast in a Live Broadcasting Project.

Support for Live H.264 VBR 1-pass (Pro only) – You can now select 1-pass VBR encoding for H.264-based Live Smooth Streaming encodes.

Other

No Screen Capture limit – The 10-minute Screen Capture limit has been removed from Expression Encoder.

Follow mouse cursor – You can now enable the screen capture area to follow your mouse cursor in Screen Capture.

Template improvements – Offline playback of DRM content, SRS playback support, improved TTML support, audio volume on startup, auto-repeat, etc.

New SDK samples – Following your requests and suggestions, we added 5 C# and 6 VB.NET samples, including a thumbnail generator, a performance tool and a Live console application.

Let us know what you think! Thanks.

So first, let's explore what's happening when encoding a media file using Expression Encoder. The encoding process involves 5 phases:

1. Reading the source(s) from storage.
2. Demuxing and decoding the source(s) into uncompressed frames.
3. Pre-processing the frames (deinterlacing, resizing, cropping, etc).
4. Encoding and muxing the frames.
5. Writing the stream to storage.

If any one of those operations is slower than the others, all the encoding pipeline will suffer in terms of performance because it will bottleneck the flow of frames into it.

#1 and #5 are usually the bottleneck when source files are either not local (aka slow Network share) or are very high-bandwidth coupled with a slow storage. Possible solutions: making sure the source files are copied locally on your fastest storage available and outputting to a different local storage are good solutions to reduce the bottleneck in those two areas.

#2 usually becomes a bottleneck because of the type of sources and the codecs used to decode them. Obviously, using complex HD sources will use significantly more resources than simple lower-resolution sources. Some 3rd party codecs are extremely slow, running only on one core. Trying to test other codecs available on the PC by disabling some of the codecs listed in the "Tools -> Options -> Compatibility" dialog may help reducing the bottleneck. It's also worth noting that because many codecs are single-thread, having faster cores can help enormeously here. This is the main reason why a 3.4MHz 4-core is faster than a 2.6MHz 8-core PC in many cases.

#3 can become the bottleneck when unneeded cropping/rescaling is applied, when "SuperSampling" resizing and/or "Auto Pixel Adaptive" deinterlacing are applied. The last two are our defaults, which were the best choice for "high quality" encodes at the time we shipped v4 RTM over 18 months ago and we unfortunately can't change the defaults until our next major version since it would be an SDK breaking change. For better balance between performance and quality, we highly recommend using "Bicubic" resizing and "Auto selective blend" deinterlacing. Both of them will perform 2-3x times faster than the defaults and will reduce the chances of the pre-processing phase being the bottleneck. Of course, if at all possible, removing the need for a resize (aka using the same frame size and pixel aspect ratio as the source) will also reduce the load on that phase while keeping better output quality.

This leaves phase #4, where most of the CPU cycles should be used if there are no bottlenecks elsewhere in the pipeline. In the best of worlds, this phase should be your bottleneck, which would very likely max out CPU usage on an 8-core PC for a single-stream encode. Depending on the encode settings, 100% may still not be achieved, but it should be pretty high. Here are few ideas to make this phase faster:

1. Make sure to use the defaults for number of threads and slices used to encode (where applicable).
2. If speed is most important, consider applying our "Fastest" Quality preset, which will significantly speed up encoding process at the cost of some output quality.
3. Use proper encode settings. For example, trying to encode 480p content into 300kbps would be stressing the encode phase and could slow it down to a crawl. We recommend about 1.5+ Mbps for 480p and about 3+ Mbps for 720p. Consider reducing the frame size and/or frame rate if you are trying to achieve significantly lower output bitrates than this.
4. If GPU encoding is enabled on a slow GPU when encoding to H.264, it could make the GPU the bottleneck. Solution: disable GPU encoding or get a better GPU.
5. On larger scale PCs (16 cores and above), single one-stream encodes can't maximize the CPU usage. Parallel encoding is an option to maximize the PC resources. If using a NUMA enabled system, process affinity is highly recommended to ensure memory is not shared across CPUs between the different encoding processes, which can reduce performance.

In the case where nothing seems to help maximizing CPU usage, one could also consider running 2-4 jobs in parallel in separate encoding processes. Using the UI, simply start 2-4 Encoder instances and run the encodes in parallel, bearing that enough system memory is available. This suggestion would certainly not help with a storage bottleneck situation (aka phase #1 and/or #5), but should greatly help speed up the process of encoding multiple jobs.

Finally, if encoding to H.264, using GPU encoding may drastically help speed up the encoding phase. Cuda is supported in Encoder 4 Pro SP1, and Intel QSV (Sandy Bridge) in our next SP release coming soon. When used with the proper hardware, either (or even both) options can cut down encoding time by more than half, and in some cases significantly more.

Hopefully, this gives you some insights on how to isolate and resolve encode performance issues.

This analysis operation, called indexing, is unfortunately time consuming since it requires to read and demux the whole media file and typically requires a few minutes per hour of content, mostly depending on storage speed. Because of this, it is recommended to first copy this type of media file locally before importing it to ensure maximum efficiency.

Note that while this operation is automatically executed upon import within the Encoder application, it is not executed by default when importing a media file via the SDK and needs to be explicitly called before the MediaItem constructor via the CalculateDuration member of AudioVideoFile like this:

                     AudioVideoFile file = new AudioVideoFile(filename);

                     file.CalculateDuration(null);

                     MediaItem item = new MediaItem(file);

It is worth pointing that if the CalculateDuration method isn't called, there will still be indexing done in-line during the encoding which will fix the timestamps, but it is possible that the duration of the output file will be wrong. For this reason, we highly recommend using the code above if source files that require indexing are used.

Microsoft exhibited with Accordent Technologies, a partner that has developed an automated software-based video creation, management, publishing, and delivery platform leveraging Microsoft infrastructure in the enterprise, including SharePoint, Lync, Windows Server, IIS, Windows Media Services, Silverlight, and Expression Encoder.

The team participated in conference panel discussions on topics ranging from mobile delivery to YouTube in the enterprise (videos of those panels coming soon), and IIS Media Services senior product manager Chris Knowlton was recorded doing both an in-booth Expression Encoder / IIS / iPad demo and an on-site interview with Steve Vonder Haar. 

Additionally, both Microsoft and Accordent were recipients of Best in Show awards - Expression Encoder was awarded Most Innovative Product and Accordent received Best Enterprise Solution.  Congrats to the team! 

Microsoft and Accordent recently participated in a webinar to talk about how the companies are addressing the challenges of deploying video in the enterprise.  To view the archived content, please login here.   

Another benefit of the EventID field is that it will allow you to restart your publishing point from a Stopped state.  When the Encoder stops broadcasting to an IIS Smooth Streaming publish point, we send an empty 'mfra' box which informs the publish point to switch to a Stopped state.  This allows for DVR playback to be available to the users immediately after the broadcast ends.  The Encoder does not allow you to connect to a publishing point that is in use and to check this, we check to see if there is a manifest associated with the pub point.  A manifest will exist if the publishing point is in a Started or Stopped state so by default we block restarting the publish point.  To work around this, if you specify a different EventID for each broadcast (including the first one, null doesn't count)  to the publishing point that contains the tag:   <meta name="restartOnEncoderReconnect" content="true" /> :  We will ignore the manifest check and restart the publishing point.  Here are the steps:

1. Install Expression Encoder 4 SP1
2. Install IIS Media 4.0 on your server
3. Create a Live Smooth Streaming publishing point on the server.
4. Using Explorer, navigate to the location of the publishing point.
5. Open the ISML file in a text editor and add the following between the <head> tags:  <meta name="restartOnEncoderReconnect" content="true" />
6. Now set up your broadcast in the Encoder and point to the publishing point.
7. Before clicking Start, specify an EventID on the Output panel.
8. Now if you want to reuse the same publishing point for another broadcast,  just specify a different EventID and click Start

• GPU encoding - If you have an NVIDIA CUDA-accelerated GPU board in your system, you can now take advantage of the additional processors to speed up encoding time or squeeze more live streams into your broadcasts.  Check out Eric Juteau's white paper for all the details.
• Screen capture as a live source - Now you can select the screen as a device in Live projects to stream or archive.  
• Live templates - Before if you wanted to use the Expression Encoder templates, you needed to hand edit the template to point to your live source.  Now you can generate the template directly from the Live project and we will hook up all the links.  The workflow is slightly different than on-demand templates.  Since these templates usually need to be live before you start broadcasting, you will need to generate the files manually beforehand.  Once you have selected your output type, the Generate button will enable on the Templates panel where you can manually create the files or upload them to a WebDAV enabled server.  If the panel doesn't appear after upgrade, you can find it on the Windows menu.
• Live Push/Pull - Now you can broadcast and publish to a Windows Media Services publish point at the same time.
• Improved De-interlacing Options - We have added a new de-interlacing option called 'Selective Blend' which provides a higher quality picture without the high CPU requirements of Pixel Adaptive.  We have also exposed this and the other de-interlacing options in live.
• Import script commands - Both in live (no timing) and on-demand, you can now import multiple captioning formats (ISMT, DFXP, SRT, XML, SAMI, SUB, TXT) to the Script Command panel.  This allows you to pre-propagate your captions in live from a script or do some basic editing of your scripts before encoding.  If you do not have a caption file for you encode or broadcast, you can now easily create a 'return-delimited' TXT file with your captions to import in either project type as well.
• DRM Content Key - If you PlayReady license server supports Content Key encryption, we now allow you to specify this key instead of a Key Seed to enable individual keys per item.
• HE-AAC - We have added both Version 1 and Version 2 of the HE-AAC codec to allow encoding for supported devices.
• New Windows Phone 7 presets - Now that the WIndows Phone 7 is out (they rock!), we have created presets to enable direct playback (WMV, MP4) or creating video to use in a Smooth Streaming application using the Silverlight Media Framework phone template.

Check it out and let us know what you think!  Thanks

Jamie

http://gallery.expression.microsoft.com/en-us/EESharePointPlugin?SRC=Home

jamie

http://bit.ly/bhTepe