Share via

Implement Hardware Acceleration for Graphics in XAML for Windows Embedded (Compact 2013)

  • Article

3/26/2014

You can use hardware acceleration for the bitmaps that are generated by the rendering process in a XAML for Windows Embedded application. By using hardware acceleration, you can offload common graphics operations from the main microprocessor onto a display controller by blitting bitmaps first to an off-screen graphical frame buffer, or surface, before they are rendered on the screen. This process creates a faster and more stable graphics environment. If you do not use hardware acceleration, Windows Embedded Compact 2013 uses Graphics Device Interface (GDI) to draw UI objects pixel-by-pixel onto the primary display surface in the order they were first loaded into the object tree during XAML parsing.

Hardware acceleration relies on a process called cached composition, which minimizes CPU usage during rasterization by using the graphics processing unit (GPU) and memory instead of the CPU. The disadvantage of this technique is that it requires additional video and system memory on the hardware board.

Graphics hardware acceleration in XAML for Windows Embedded is based on one of the following:

  • DirectDraw   Supports hardware-accelerated two-dimensional graphics. For more information, see DirectDraw in the Windows Embedded Compact 2013 Help.

  • OpenGL   Supports hardware-accelerated two-dimensional or three-dimensional graphics, skewing, rotation, and plane projection. For more information, see Introduction to OpenGL on MSDN.

    Note

    If you implement hardware acceleration based on OpenGL, the XAML UI that is rendered on the screen has a maximum size limitation of 2,048 × 2,048 pixels.

This tutorial contains the following major steps.

  • Step 1: Add Support for Hardware Acceleration to the OS Design
    Use this procedure to add support for hardware acceleration to your OS design by including BSP flags. If you are not going to customize the rendering plug-ins, you can proceed to Step 6.
  • Step 2: Customize the Hardware Configuration and Graphics-Rendering Behavior
    Use this procedure to learn how to customize the graphics renderer that XAML for Windows Embedded uses when it displays graphics. You do not have to customize the graphics renderer unless you want to maximize the performance of the renderer code on your specific device hardware.
  • Step 3: Customize the DrawTriangleStrip Method
    Use this procedure to learn how to customize the DrawTriangleStrip method. DrawTriangleStrip is called internally to draw a series of triangles by using a set of input vertices that are provided in an array of XRVertex structures. DrawTriangleStrip is part of the rendering plug-in.
  • Step 4: Build Your Code
    If you customized the rendering plug-in, use this procedure to build the DLL.
  • Step 5: Include the Security Model
    Use this procedure to help make sure that access to your customized DLL is secure by including the trusted security model and implementing load privilege for XamlRenderPlugin.dll.
  • Step 6: Build the OS Design into a Run-Time Image
    Use this procedure to create a run-time image that includes support for hardware acceleration and your customized DLL if you created one.
  • Step 7: Use Hardware Acceleration in Your Application
    Use this procedure to learn how to cache bitmaps and use the z-order of UI elements to optimize hardware acceleration.

Prerequisites

Before you begin this tutorial, make sure that:

Step 1: Add Support for Hardware Acceleration to the OS Design

You support hardware acceleration in an OS design by adding a board support package (BSP) flag. For example, after you add the BSP flag for DirectDraw, you can customize the DirectDraw plug-in (see Step 2: Customize the Hardware Configuration and Graphics-Rendering Behavior through Step 7: Use Hardware Acceleration in Your Application below), or just use hardware acceleration in your application (see Step 6: Build the OS Design into a Run-Time Image and Step 7: Use Hardware Acceleration in Your Application below).

To set a BSP flag at the command line

  1. In Platform Builder, on the Build menu, click Open Release Directory in Build Window.

  2. To see if one of the plug-ins is already set, type Set | findstr BSP_XRPLUGIN. If nothing is returned, you can continue with the next step.

  3. At the command prompt, type Set, type a space, and then type the flag you want to set. For example:

    • To add the DirectDraw rendering plug-in to your OS design, type Set BSP_XRPLUGIN_DDRAW=1.
    • To add the OpenGL rendering plug-in to your OS design, type Set BSP_XRPLUGIN_OPENGL=1.

If you are not going to customize the rendering plug-in you chose, proceed to Step 6.

Step 2: Customize the Hardware Configuration and Graphics-Rendering Behavior

You can customize the graphics renderer that is used internally by XAML for Windows Embedded when it displays graphics. You do not have to customize the graphics renderer unless you want to maximize the performance of the renderer code on your specific device hardware.

Before you customize the graphics renderer, it is a good idea to back up the source code files by using a source code control system.

The default DirectDraw plug-in is implemented in:

  • IRenderer   To customize, implement your changes in %_WINCEROOT%\Public\Common\Oak\XamlRenderPlugin\Ddraw\Ddrawrenderer.cpp.
  • ICustomSurface   To customize, implement your changes in %_WINCEROOT%\Public\Common\Oak\XamlRenderPlugin\Ddraw\Ddrawsurface.cpp.
  • ICustomGraphicsDevice   To customize, implement your changes in %_WINCEROOT%\Public\Common\Oak\XamlRenderPlugin\Ddraw\Ddrawdevice.cpp.

The default OpenGL plug-in is implemented in:

  • IRenderer   To customize, implement your changes in %_WINCEROOT%\Public\Common\Oak\XamlRenderPlugin\Opengl\Openglrenderer.cpp.
  • ICustomSurface   To customize, implement your changes in %_WINCEROOT%\Public\Common\Oak\XamlRenderPlugin\Opengl\Openglsurface.cpp.
  • ICustomGraphicsDevice   To customize, implement your changes in %_WINCEROOT%\Public\Common\Oak\XamlRenderPlugin\Opengl\Opengldevice.cpp.

To customize the DirectDraw or OpenGL graphics renderer

  1. In Platform Builder, open your OS design project.

  2. In Solution Explorer, browse to <OS Design Name>\ C:\WINCE800\Public\Common\Oak\XamlRenderPlugin.

  3. Expand either DDraw or OpenGL, depending on which plug-in you enabled in Step 1.

  4. Browse to the .cpp file that you want to customize, and double-click the .cpp file. The files may include comments that help identify where code can be customized. For example, in %_WINCEROOT%\Public\Common\Oak\XamlRenderPlugin\Opengl\Opengldevice.cpp, there is the following comment in the COpenGLDevice::InitializeState() function:

        // Initialize the shaders
    // If we can't load binary shaders that match this hardware,
    // the following source will be compiled instead.
    // But you should really provide optimized binary shaders.

    CHR(InitializeShader(GL_VERTEX_SHADER, L"DefaultEffect.vs", 
    ...
    CHR(InitializeShader(GL_FRAGMENT_SHADER, L"DefaultEffect.fs", 
    ...
        "   // This is worth removing if you know your hardware does BGRA;\r\n"
        "   // even the if() test can be expensive.\r\n"
    ...

  5. If you are customizing the OpenGL renderer, consider defining the following flags to include the related features:

    • USE_ANITIALIASING   Antialiasing refers to the smoothing of jagged edges of drawn graphics and text to improve their appearance or readability.
    • USE_DEPTH_BUFFER   A depth buffer contains per-pixel floating-point data for the z-depth of each pixel rendered. When a pixel is rendered, color data in addition to depth data can be stored. If a pixel is rendered a second time--such as when two objects overlap-depth testing determines which pixel is closer to the camera and, thus, which pixel value will be stored and drawn.
    • USE_BACKFACE_CULLING   A backface is a triangle whose normal faces away from the camera. For example, if you render a cube, there is no need to render all of the triangles that are on the back side of the cube because they are not visible to the user.

  6. After you finish customizing the code in the .cpp files, save your changes.

To customize the source code for the interfaces, you can add code to the following default method implementations, which are called internally by XAML for Windows Embedded when it displays graphics.

IRenderer

Methods

Description

CreateRenderer

Creates an IRenderer object for the host window. If the plug-in supports cached composition of bitmaps, this method also creates an ICustomGraphicsDevice object.

IRenderer::FreeResources

Frees resources that are associated with the off-screen surface and the ICustomGraphicsDevice object.

IRenderer::PreRender

For the DirectDraw renderer, creates an off-screen surface for blitting the graphics before they are displayed on the screen. You can add code inside this method to perform rendering tasks that are required before the UI can be displayed on the screen.

IRenderer::PostRender

Blits the off-screen surface to the primary surface. This method adds code to complete all work that is needed after rendering a scene.

RenderPluginInitialize

Initializes the rendering plug-in.

RenderPluginCleanup

Cleans up allocated resources when XAML for Windows Embedded exits.

ICustomSurface

Methods

Description

ICustomSurface::Lock

Obtains a pointer to the surface that is allocated in system memory, and retrieves the dimensions, in pixels, of the surface to be created.

ICustomSurface::Unlock

Unlocks the surface that was previously locked.

ICustomSurface::Present

Dumps the content of the back buffer onto the screen.

For DirectDraw, the content of the back buffer is blitted on to the primary surface that is associated with the client window.

For OpenGL, you can use eglSwapBuffer() to post the content of the target buffer to the associated window.

ICustomSurface::GetWidth

Retrieves the width, in pixels, of the surface.

ICustomSurface::GetHeight

Retrieves the height, in pixels, of the surface.

ICustomSurface::GetPixelFormat

Retrieves the color and pixel format of the surface.

ICustomSurface::IsVideoSurface

Indicates whether the surface contains video content.

ICustomSurface::IsOpaque

Returns 1 if the custom surface is opaque. Otherwise, returns 0 (zero).

ICustomSurface::IsTransparent

Returns 1 if the custom surface is transparent. Otherwise, returns 0 (zero).

ICustomSurface::SetIsOpaque

Sets whether the custom surface is opaque.

ICustomSurface::SetIsTransparent

Sets whether the custom surface is transparent.

ICustomGraphicsDevice

Programming element

Description

ICustomGraphicsDevice::Clear(UINT uColor)

Clears the back buffer content by setting it to the color that is specified in uColor.

For OpenGL, clears and fills its color buffer by calling glClearColor().

For the DirectDraw plug-in, fills the back buffer with the color in uColor by using the Blt function.

ICustomGraphicsDevice::Initialize(HWND hWindow)

Sets up the primary surface that is related to the client window.

ICustomGraphicsDevice::Resize(UINT uWidth, UINT uHeight)

Updates all related structures or data after the client window size is changed. This method is called when a window size changes.

For DirectDraw, a surface of the new size must be created.

For OpenGL, the viewport and uniform matrix are updated.

ICustomGraphicsDevice::CreateTexture( Int fRenderTarget, UINT nWidth, UINT nHeight, Int fKeepSystemMemory, ICustomSurface **ppSurface)

Provides an entry point for creating customized surfaces.

The fRenderTarget parameter specifies whether the surface is the target surface that is used to buffer graphics before displaying them on the screen. The fRenderTarget parameter is currently only used in the DirectDraw-based renderer.

The fKeepSystemMemory parameter specifies whether this surface must be kept as a copy in system memory. When this parameter is true, you must allocate a surface of the same size in system memory for performing rasterization and updates.

For DirectDraw, the surface is a DirectDraw surface of type DDSCAPS_SYSTEMMEMORY.

For OpenGL, this method allocates a section of memory on the heap.

ICustomGraphicsDevice::SetTexture( UINT uSampler, ICustomSurface *pTexture)

Sets the specified ICustomSurface object, pTexture, as the texture surface to be rendered. Call this method before you call DrawTriangleStrip().

For DirectDraw, this method keeps this surface as the source surface to be drawn.

For OpenGL, this method sets this texture surface to the drawing context.

ICustomGraphicsDevice::SetRenderTarget( ICustomSurface *pRenderTarget )

Sets the surface in the pRenderTarget parameter as the target surface. Currently this method is only used in the DirectDraw based renderer.

ICustomGraphicsDevice::DrawTriangleStrip( _ecount(cVertices) XRVertex *pVertices, UINT cVertice)

Draws a series of triangles by using the specified set of input vertices that are provided in an array of XRVertex structures.

This is the worker method for cached composition. The XRVertex structure includes the destination surface coordinates, texture coordinates, and diffuse color. Each pixel is multiplied by the diffuse color, including the alpha component. In this way, XAML for Windows Embedded can apply additional transparency or opacity to the bitmap.

For OpenGL, this method defines the vertex attributes for vertex position, diffuse, and texture by calling glVertexAttribPointer() before it calls glDrawArrays(GL_TRIANGLE_SRIP) to draw the array of vertices.

For DirectDraw, the plug-in calculates the destination and source rectangle according to the vertex coordinates and calls AlphaBlt() to draw the triangles. However, DrawTriangleStrips in DirectDraw does not support vertices of more than 4 nor does it support rotation and skewing. Instead, it falls back to software rendering.

ICustomGraphicsDevice::Present()

Presents the content of the back buffer on the screen.

For DirectDraw, blits the content in the target surface onto the primary surface.

For OpenGL, uses eglSwapBuffer() to post the buffer to the associated window.

ICustomGraphicsDevice::GetTextureMemoryUsage

Retrieves memory usage for the texture surface.

ICustomGraphicsDevice::IsHardwareComposited

Indicates whether the hardware meets the requirement of cache composition. If it does not, XAML for Windows Embedded does not follow the cached composition path, even if the source XAML has cached elements in it.

Step 3: Customize the DrawTriangleStrip Method

The renderer calls the DrawTriangleStrip method to draw a series of triangles by using a set of input vertices that are provided in an array of XRVertex structures. To customize the DrawTriangleStrip method, modify one of the following files:

  • For OpenGL, the COpenGLDevice DrawTriangleStrip method is located in %_WINCEROOT%\Public\Common\Oak\XamlRenderPlugin\Opengl\Opengldevice.cpp.
  • For DirectDraw, the CDDrawDevice DrawTriangleStrip method is located in %_WINCEROOT%\Public\Common\Oak\XamlRenderPlugin\Ddraw\Ddrawdevice.cpp.

Note

For DirectDraw, the element is rendered when the set of four vertices represent a rectangle. The rectangle is drawn onto the surface in the form of two half-triangles within the boundaries of that rectangle. If cVertices is not equal to 4, the system uses default software rendering.

An XRVertex structure defines a vertex, which is the point at which two sides of an angle intersect. The following table lists the members of the XRVertex structure.

Member

Description

XRVertex.x

The x-coordinate of the vertex.

XRVertex.y

The y-coordinate of the vertex.

XRVertex.z

The z-coordinate of the vertex, which you can use only for three-dimensional rendering. This value must be the same for each vertex.

XRVertex.u0

The x texture-coordinate, which you use for texture-stage 0.

XRVertex.v0

The y texture-coordinate, which you use for texture-stage 0.

XRVertex.u1

The x texture-coordinate, which you use for texture-stage 1.

XRVertex.v1

The y texture-coordinate, which you use for texture-stage 1.

XRVertex.dwDiffuse

The opacity value.

You can assign texture coordinates directly to vertices by using the u0, v0, u1, and v1 members. A texture is a bitmap of pixel colors that give an object the appearance of texture. Texture can give visual depth to an object, such as applying the texture of sandpaper to a beige background to make it appear rough. By assigning texture coordinates directly to vertices, you can control which part of a texture is mapped onto a primitive.

A texture can be assigned to a stage so that you can define how that texture is rendered. For example, you can blend multiple textures together. By using XRVertex, you can change how the textures appear in different areas by applying texture coordinates for two different texture stages.

Possible effects include changing texture filling, environment mapping, or adjusting the level of detail.

Step 4: Build Your Code

If you customized the rendering plug-in in Step 2 or Step 3, use this procedure to build the DLL.

To recompile and build the graphics renderer

  1. In Solution Explorer, browse to <OS Design Name>\C:\WINCE800\Public\Common\Oak\XamlRenderPlugin.

  2. Right-click either DDraw or OpenGL and click Rebuild.

  3. In the Output window, verify that the build succeeded by looking for the following output message:

    ========== Build: 1 succeeded or up-to-date, 0 failed, 0 skipped ==========

  4. Create an updated run-time image that includes the rebuilt graphics renderer.

    1. On the Build menu, click Copy Files to Release Directory.
    2. On the Build menu, click Make Run-Time Image.
    3. In the Output window, verify that the build succeeded.

If BSP_XRPLUGIN_OPENGL is set, Windows Embedded Compact renames XRRendererOpengl.dll to XamlRenderPlugin.dll during the build process. If BSP_XRPLUGIN_DDRAW is set, Windows Embedded Compact renames XRRendererDDraw.dll to XamlRenderPlugin.dll during the build process. If none is set, no XamlRenderPlugin.dll is generated; therefore, XAML for Windows Embedded uses an internal GDI renderer. Customers may provide their own implementation of XamlRenderPlugin.dll.

In your DLL, you must export the following names:

  • RenderPluginInitialize
  • RenderPluginCleanup
  • CreateRenderer

You must implement your DLL to return S_OK when XAML for Windows Embedded calls RenderPluginInitialize.

Step 5: Include the Security Model

To help make sure that access to the DLL is secure, include the trusted security model and implement load privilege for XamlRenderPlugin.dll.

To include the security model

  1. Add the Loader Verifier Module (LVMOD) and API Sysgen variable (SYSGEN_LVMOD) to the OS image.

    For more information about SYSGEN_LVMOD, see Security Loader Catalog Items and Sysgen Variables.

  2. Add one or more certificates to the Code Integrity certificate store on the device.

    For more information about how to add certificates, see Adding Certificates to the Code Integrity Store.

  3. Ensure that the graphics-rendering DLL is either stored in ROM or signed with a certificate that is in the Code Integrity certificate store on the device.

    For more information about how to sign the graphics-rendering DLL with a certificate, see Signing Binaries.

These steps help prevent a malicious user from installing an unauthentic XamlRenderPlugin.dll that has malicious software that compromises all XAML for Windows Embedded applications on the device.

Otherwise, do not allow non-trusted applications to be installed on the device.

Step 6: Build the OS Design into a Run-Time Image

Build the OS design project and the run-time image, and download it to the device through the connection that you have already configured. For information about how to run your OS image on a virtual CEPC, see Use the Sample Virtual Device.

If you already have a run-time image that includes the BSP flags from Step 1, and you customized the graphics renderer, you can copy the DLL for the renderer to the release directory, which is located at %_WINCEROOT%\<OSDesign>\<OSDesign>\RelDir.

Step 7: Use Hardware Acceleration in Your Application

In your XAML for Windows Embedded application, to use hardware acceleration, you must set the cache mode (CacheMode) to bitmap caching ("BitmapCache") only for UI elements that use hardware acceleration at run time. You can set cache mode either in the source XAML file or in the C++ application.

We recommend that you set the cache mode for all UI elements for which you implement transformations or animations. Both of these require additional GPU processing.

To set the cache mode for a UI element

  1. Do one of the following:

    • In your XAML file, in the element, add a CacheMode attribute and set its value to "BitmapCache," as shown in the following code example:

      <Image x:Name="_Globe" 
    Height="138" HorizontalAlignment="Center" Margin="0,20,90,0" VerticalAlignment="Top" Width="138"
    Source="Assets/Globe138x138.png" Stretch="Fill" RenderTransformOrigin="0.5,0.5" 
    CacheMode="BitmapCache">

    • In your application, locate the UI element in the visual tree by using the IXRFrameworkElement::FindName method, create an IXRBitmapCache object (using the corresponding smart pointer IXRBitmapCachePtr), and then pass the IXRBitmapCache object into IXRUIElement::SetCacheMode.

      IXRListBoxPtr pElement;
pRoot->FindName(L"FancyListBox", &pElement);
IXRBitmapCachePtr pCache;
pApplication->CreateObject(&pCache);
pElement->SetCacheMode(pCache);

You can reduce graphics memory consumption during graphics rendering by changing the default z-order of UI elements and by positioning all cached elements higher in the z-order. We recommend that you place UI elements that use bitmap caching, such as animations, first in the z-order. When these UI elements are rendered first, it reduces the amount of back-buffer surfaces that the system must create for other elements during the rendering process, which reduces graphics memory consumption.

To specify z-order for a UI element

  1. Do one of the following:

Design Guidelines

When you design a UI to take advantage of GPU-based hardware acceleration, make sure that you take advantage of the bitmap cache and element z-order as previously described The following general guidelines offer additional suggestions for getting the most from GPU-based hardware acceleration. More detailed guidelines are available in the Graphics and Performance in XAML for Windows Embedded section.

General Guidelines

  • Use UI elements that are optimized for hardware acceleration.
    For information, see Optimize UI Elements and XAML Files for Graphics Performance.
  • Limit the XAML UI to a maximum size of 2,048 × 2,048 pixels for OpenGL applications.
    OpenGL has a limitation that surfaces cannot be larger than 2,048 × 2,048. If you attempt to create an OpenGL surface that is larger than 2,048 × 2,048 pixels, it will appear as a black rectangle on the screen. XAML for Windows Embedded will not automatically split a large surface into multiple smaller ones, so it is important to consider this limitation when you design for very large screens.
  • Limit large-scale animations to transformations that can be hardware-accelerated.
    XAML for Windows Embedded permits many attributes to be animated, but only a few of these can be hardware-accelerated. Examples of animations that cannot be accelerated include repositioning an item by changing margins and animating gradients and colors. Performing these types of animations on a small part of the screen may not cause any performance difficulties but, for example, animating a screen-sized radial gradient is almost certain to overtax the CPU. Note that if a UI element is changing during an animation, it is repeatedly re-rendered, negating the benefits of hardware acceleration.
  • Use the Visibility or Opacity properties to efficiently hide or display elements in your UI.
    While an object is hidden (visibility = collapsed), XAML for Windows Embedded does not hold visual data for the object in video memory, it does not walk the tree in the render pass, and it does not do any hit-test work. When the object is visible again (visibility = visible), the entire tree content has to be drawn again (re-layout). If the object is cached, setting opacity to zero to hide the object causes XAML for Windows Embedded to hold a texture for that tree in video memory without affecting the performance of the compositor. When the object is visible again (opacity > 0) it will be processed normally without redrawing the content. You will likely obtain the best results by manipulating the opacity property, but with multiple or complex trees, using the visibility property may be the best approach. Try both approaches and see which works best for you.

XAML Structure Guidelines

You can also optimize rendering by structuring the XAML to improve buffer usage. The following list shows some examples to illustrate this guideline:

  • If there is a set of static elements that serve as a background, group them and put them at the start of the XAML, so that they will all be placed in the same buffer in the GPU.
  • Make sure that dynamic items are not grouped with a set of static elements. Using the button example, if you want the button to draw attention by gradually changing its brightness, make sure that it is not in the same buffer as the static background items. Otherwise, each brightness change would require that the entire background be rasterized and composed again.
  • Do not unnecessarily create extra buffers. Buffers use memory in both the CPU and GPU, so extra buffers must not be created without reason. In our example, _Globe is placed at the end of the XAML, and only two buffers are needed. However, if _Globe was placed earlier in the XAML, the background items would be rendered in two separate buffers, meaning that three buffers would be used.

See Also

Concepts

Hardware Acceleration in XAML for Windows Embedded