Share via

Implement Hardware Acceleration for Graphics in Silverlight for Windows Embedded (Windows Embedded CE 6.0)

  • Article

1/6/2010

You can create a faster and more stable graphics environment for bitmaps that were created with Silverlight for Windows Embedded by offloading common graphics operations from the main microprocessor onto a display controller. To do this, you blit bitmaps, which is the process of transferring the bitmaps, first to an off-screen graphical frame buffer, or surface, before they are rendered on the screen.

Graphics hardware acceleration in Silverlight is based on one of the following:

  • DirectDraw   Supports hardware-accelerated two-dimensional graphics. For more information, see DirectDraw and Why Use DirectDraw?.
  • OpenGL   Supports hardware-accelerated two-dimensional or three-dimensional graphics, skewing, and rotation. For more information, see Introduction to OpenGL on MSDN.

Hardware acceleration relies on a process called cached composition, which minimizes CPU usage during rasterization by using the graphics processing unit (GPU) and memory instead of the CPU. The downside of this technique is that it requires additional video and system memory.

If you do not implement hardware-accelerated graphics, Windows Embedded CE uses Graphics Device Interface (GDI) to draw UI objects pixel-by-pixel onto the primary display surface in the order that they were first loaded into the object tree during XAML parsing. For more information, see Graphics Device Interface (GDI).

Prerequisites

In order to use hardware acceleration, the application and the device upon which it runs must meet the following requirements:

  • A Silverlight application. For more information, see Create a Silverlight for Windows Embedded Application.

  • A standard development board that meets the following suggested guidelines:**

    • ARM V4 micro-processor
    • 550 MHZ processor
    • 256 MB RAM
    • Two-dimensional Graphics Processing Unit (GPU) with DirectDraw or OpenGL interface**

  • For DirectDraw, a GPU that supports the following:

    • Per-pixel and constant alpha blits
    • Premultiplied alpha blit
    • Hardware-accelerated alpha blits
    • Hardware-accelerated blits on SRCCOPY raster operations between system video and video memory
    • Hardware-accelerated stretch blits for SRCCOPY raster operations
    • Hardware-accelerated color fill **

  • For OpenGL, an OpenGL driver that supports the OpenGL Embedded Systems (ES) 2.0 specification, which supports the following:

    • Passes the Khronos conformance tests. For more information, see the Khronos website.
    • Supports a simple vertex and fragment shader. Silverlight uses these shaders to draw textured rectangles on the screen. You can find sample code for these shaders in %_WINCEROOT%\public\COMMON\oak\xamlrenderplugin\OpenGL\Shaders.
      The vertex and fragment shaders must be compiled for the platform's GPU, and linked into a file named Shaders.dll. Windows Embedded CE 6.0 does not include a command line shader compiler; you must obtain one from your hardware manufacturer. If CE 6.0 cannot find the Shaders.dll library for the platform’s GPU it tries to compile the default shaders at run time. However, many OpenGL drivers will not support run-time compilation, and compiling the shaders at run time can result in poor performance. Consult your hardware provider for instructions on compiling shaders for the target GPU.

Step 1 Add Support for Hardware Acceleration to the OS Design

To add the DirectDraw rendering plug-in and the OpenGL rendering plug-in, include the SYSGEN variable SYSGEN_XAML_RUNTIME. This SYSGEN variable will set the Board Support Package (BSP) flags BSP_XRPLUGIN_DDRAW and BSP_XRPLUGIN_OPENGL. For more information, see Environment Variables.**

Step 2 Implement Your Hardware Configuration and Graphics-Rendering Behavior

The default DirectDraw plug-in is implemented in the following interfaces:

  • IRenderer   To customize, implement your changes in %_WINCEROOT%\PUBLIC\COMMON\OAK\XAMLRENDERERPLUGIN\DDRAW\ ddrawrenderer.cpp.
  • ICustomSurface   To customize, implement your changes in %_WINCEROOT%\PUBLIC\COMMON\OAK\XAMLRENDERERPLUGIN\DDRAW\ ddrawsurface.cpp.
  • ICustomGraphicsDevice   To customize, implement your changes in a copy of %_WINCEROOT%\PUBLIC\COMMON\OAK\XAMLRENDERERPLUGIN\DDRAW\ddrawdevice.cpp.

The default OpenGL plug-in is implemented in the following interfaces:

  • IRenderer   %_WINCEROOT%\PUBLIC\COMMON\OAK\XAMLRENDERERPLUGIN\OPENGL\openglrenderer.cpp
  • ICustomSurface   %_WINCEROOT%\PUBLIC\COMMON\OAK\XAMLRENDERERPLUGIN\OPENGL\openglsurface.cpp
  • ICustomGraphicsDevice   %_WINCEROOT%\PUBLIC\COMMON\OAK\XAMLRENDERERPLUGIN\OPENGL\opengldevice.cpp

Add code to the following methods, which are called by Silverlight when it displays graphics.

Ee504451.collapse(en-US,WinEmbedded.60).gifIRenderer

Programming element Description

IRenderer::FreeResources

Frees resources associated with the off-screen surface and the ICustomGraphicsDevice object.

IRenderer::PreRender

For the DirectDraw renderer, creates an off-screen surface for blitting the graphics before they are displayed on the screen. Add code to complete all the work that needs to be done before a scene can be rendered on the screen.

IRenderer::PostRender

Blits the off-screen surface to the primary surface. Add code to complete all work needed after rendering a scene.

RenderPluginInitialize

Initializes the rendering plug-in.

RenderPluginCleanup

Cleans up allocated resources when Silverlight shuts down.

CreateRenderer

Creates an IRenderer object for the host window. If the plug-in supports cached composition of bitmaps, CreateRenderer also creates an ICustomGraphicsDevice object.

Ee504451.collapse(en-US,WinEmbedded.60).gifICustomSurface

Programming Element Description

ICustomSurface::Lock

Obtains a pointer to the surface allocated in system memory and retrieves the dimensions, in pixels, of the surface to be created.

ICustomSurface::Unlock

Unlocks the surface that was previously locked.

ICustomSurface::Present

Dumps the content of the back buffer onto the screen.

For DirectDraw, the content of the back buffer is blitted on to the primary surface associated with the client window.

For OpenGL, you use eglSwapBuffer() to post the content of the target buffer to the associated window.

ICustomSurface::GetWidth

Retrieves the width, in pixels, of the surface.

ICustomSurface::GetHeight

Retrieves the height, in pixels, of the surface.

ICustomSurface::GetPixelFormat

Retrieves the color and pixel format of the surface.

ICustomSurface::IsOpaque

Returns 1 if the custom surface is opaque. Otherwise, returns 0 (zero).

ICustomSurface::IsTransparent

Returns 1 if the custom surface is transparent. Otherwise, returns 0 (zero).

ICustomSurface::SetIsOpaque

Sets whether the custom surface is opaque.

ICustomSurface::SetIsTransparent

Sets whether the custom surface is transparent.

Ee504451.collapse(en-US,WinEmbedded.60).gifICustomGraphicsDevice

Programming Element Description

ICustomGraphicsDevice::Initialize(HWND hWindow)

Sets up the primary surface that is related to the client window.

ICustomGraphicsDevice::Resize(UINT uWidth, UINT uHeight)

Called when the client window size changes. When the size changes, all the related structures or data must be updated.

For DirectDraw, a surface of the new size must be created.

For OpenGL, the viewport and UniformMatrix will be updated.

ICustomGraphicsDevice::CreateTexture(

Int fRenderTarget,

UINT nWidth,

UINT nHeight,

Int fKeepSystemMemory,

ICustomSurface **ppSurface

)

Entry point for creating customized surfaces.

The fRenderTarget parameter specifies whether the surface is the target used to buffer graphics before displaying them on the screen. This parameter is currently used only in the DirectDraw-based renderer.

The fKeppSystemMemory parameter specifies whether this surface must be kept as a copy in system memory. When this parameter is true, you have to allocate a surface of the same size in system memory to perform rasterization and updates.

For DirectDraw, the surface is a DirectDraw surface of type DDSCAPS_SYSTEMMEMORY.

For OpenGL, you allocate a section of memory on the heap.

ICustomGraphicsDevice::SetTexture(

UINT uSampler,

ICustomSurface *pTexture)

Sets the specified ICustomSurface object, pTexture, as the texture surface to be rendered. Call this method before calling DrawTriangleStrip().

For DirectDraw, you keep this surface as the source surface to be drawn.

For OpenGL, you bind this texture surface to the drawing context.

ICustomGraphicsDevice::SetRenderTarget(

ICustomSurface *pRenderTarget

)

Sets the surface in the pRenderTarget parameter as the target surface.

Currently, this method is used only in the DirectDraw-based renderer.

ICustomGraphicsDevice::DrawTriangleStrip(

_ecount(cVertices) XRVertex *pVertices,

UINT cVertice

)

Draws a series of triangles by using the specified set of input vertices provided in an array of XRVertex structures.

DrawTriangleStrip is the worker function for cached composition. The XRVertex structure includes the destination surface coordinates, texture coordinates, and diffuse color. Each pixel must be multiplied by the diffuse color, including the Alpha component. In this way, Silverlight can apply additional transparency or opacity to the bitmap.

For OpenGL, you define the vertex attributes for vertex position, diffuse, and texture by calling glVertexAttribPointer() before it calls glDrawArrays(GL_TRIANGLE_SRIP) to draw the array of vertices.

For DirectDraw, the plug-in calculates the destination and source rectangle according to the vertex coordinates and calls AlphaBlt() to draw the triangles. However, DrawTriangleStrips in DirectDraw will not support vertices of more than 4 nor rotation and skewing. It will rely on software rendering in those cases.

ICustomGraphicsDevice::Clear(UINT uColor)

Clears the back buffer content by setting it to the color specified in uColor.

For OpenGL, you clear and fill its color buffer by calling glClearColor().

For DirectDraw, the plug-in fills the back buffer with the color in uColor by using the Blt function.

ICustomGraphicsDevice::Present()

Presents the contents of the back buffer on the screen.

For DirectDraw, you blit the content in the target surface onto the primary surface.

For OpenGL, you use eglSwapBuffer() to post the buffer to the associated window.

ICustomGraphicsDevice::GetTextureMemoryUsage

Retrieves the memory usage for the texture surface.

ICustomGraphicsDevice::IsHardwareComposited

Indicates whether the hardware meets the requirement of cache composition. If it does not, Silverlight will not follow the cached composition path, even if the source XAML has cached elements in it.

Step 3 Customize DrawTriangleStrip

DrawTriangleStrip draws a series of triangles by using a set of input vertices provided in an array of XRVertex structures.

For DirectDraw:

  • CDDrawDeviceDrawTriangleStrip method is located in %_WINCEROOT%\PUBLIC\COMMON\OAK\XAMLRENDERERPLUGIN\DDRAW\ddrawdevice.cpp.**

For OpenGL:

  • COpenGLDeviceDrawTriangleStrip method is located in %_WINCEROOT%\PUBLIC\COMMON\OAK\XAMLRENDERERPLUGIN\OPENGL\opengldevice.cpp.**

Note   For DirectDraw, the element is rendered when the set of four vertices represent a rectangle. The rectangle is drawn onto the surface in the form of two half-triangles within the boundaries of that rectangle. If cVertices is not equal to 4, the system uses default software rendering.

A XRVertex structure defines a vertex, which is the point at which two sides of an angle intersect. The table below shows the following XRVertex members.

Member Description

XRVertex.x

The x-coordinate of the vertex.

XRVertex.y

The y-coordinate of the vertex.

XRVertex.z

The z-coordinate of the vertex. Reserved.

XRVertex.u0

The x texture-coordinate used for texture-stage 0 (zero). Not used..

XRVertex.v0

The y texture-coordinate used for texture-stage 0 (zero). Not used.

XRVertex.u1

The x texture-coordinate used for texture-stage 1.

XRVertex.v1

The y texture-coordinate for texture-stage 1.

XRVertex.dwDiffuse

The opacity value.

You can assign texture coordinates directly to vertices by using the u0, v0, u1, and v1 members. A texture is a bitmap rendered by Silverlight.

Step 4 Build Your Code

If BSP_XRPLUGIN_OPENGL is set, Windows Embedded CE renames XRRendererOpengl.dll to XamlRendererPlugin.dll during the build process. If BSP_XRPLUGIN_DDRAW is set, Windows Embedded CE renames XRRendererDDraw.dll to XamlRendererPlugin.dll during the build process. If none is set, no XamlRendererPlugin.dll is generated so Silverlight uses an internal GDI renderer. You may provide your own implementation of XamlRenderPlugin.dll. In your DLL, you must export the following names:

  • RenderPluginInitialize
  • RenderPluginCleanup
  • CreateRenderer

You must implement your DLL to return S_OK when Silverlight calls RenderPluginInitialize.

Step 5 Include the Security Model

To help ensure that access to the DLL is more secure, include the trusted security model and implement load privilege for XamlRenderPlugin.dll.

To include the security model

  1. Enable SYSGEN_CERTMOD in the image. You can set this SYSGEN variable at the command line during a build.

  2. Add one or more certificates to the codesign store on the device.

  3. Ensure that the graphics rendering DLL is either stored in ROM or signed with a certificate that is in the codesign store on the device.

Adding a certificate to the codesign store on the device helps prevent an attacker from installing an unauthentic XamlRenderPlugin.dll that has malicious code that compromises all Silverlight applications on the device.

Otherwise, you should not allow nontrusted applications to be installed on the device.

For more information about Load Privilege, see Windows Embedded CE 6.0 Security Model.

Step 6 Build the OS Design

If you already have a run-time image, you can copy the DLL to the release directory, which is located under %WINCEROOT_%\OSDesigns\<OS design>\RelDir.

Otherwise, build your OS design. For more information, see Building a Run-Time Image for a Target Device.

Step 7 Use Hardware Acceleration in Your Application

In your application, to use hardware acceleration, you must set the cache mode to bitmap-caching only for UI elements that will use hardware-acceleration at run time. You can set cache mode in either the source XAML file or in the C++ application.

To set the cache mode for a UI element

  • Do one of the following:

    • In your XAML file, in the element, add a CacheMode attribute and set its value to "BitmapCache."

      <Rectangle Fill=”#7FFF0000” Stroke=”Black” With=”200” Height=”100” CacheMode=”BitmapCache”/>

    • In your application, locate the UI element in the visual tree, create an IXRBitmapCache, and then pass the IXRBitmapCache into IXRUIElement::SetCacheMode as shown below.

      IXRListBox* pElement;
pRoot->FindName(L"FancyListBox", &pElement);
IXRBitmapCache* pCache;
pApplication->CreateObject(IID_IXRBitmapCache, &pCache);
pElement->SetCacheMode(&pCache);

You can also reduce graphics memory consumption during graphics rendering by changing the default z-order of UI elements and positioning all cached elements higher in the z-order.

In the Silverlight API, application developers can specify a different z-order value in the Canvas.ZIndex attached property by calling IXRDependencyObject::SetAttachedProperty(const WCHAR*, int) on a UI element.

In Microsoft Silverlight 2 XAML, designers can specify z-order as the value of the Canvas.ZIndex attribute in a XAML element.

See Also

Concepts

Silverlight for Windows Embedded