UIElementRendererHelperSample.zip

With the new Windows Phone Developer Tools 7.1 Beta you can combine Silverlight and XNA Framework in a single app for Windows Phone. This is super powerful for enabling rich UI for games or for rendering 3D content for your application (among an almost infinite list of other ideas I'm sure you'll all come up with). The feature is very powerful, but I wanted to make it even easier for developers.

Built into the framework is the new UIElementRenderer type. This new object takes a Silverlight UIElement and renders it to an XNA Framework Texture2D object. You then can draw that Texture2D to the screen using the XNA Framework APIs (like SpriteBatch) which gives you Silverlight UI integrated into your XNA Framework rendered content. However there are a couple interesting things we found during development:

1. There is a lot of boiler plate code when you desire to render the whole Silverlight page, such as hooking the LayoutUpdated event, calling Render on the UIElementRenderer, and the SpriteBatch code to draw it to the screen.
2. Handling the TextBox control properly with the on screen keyboard is a bit of a chore, since you need to watch the page transform and adjust your rendering of the page directly.

Given these two concerns I cooked up a class called the UIElementRendererHelper. What the helper does is allow you to add full page rendering with UIElementRenderer, and handle all the page transform stuff, in just three lines of code for your page. Here's the basic usage:

// Make a field for the helper
UIElementRendererHelper uiRenderer;

// Construct after enabling XNA rendering in OnNavigatedTo
uiRenderer = new UIElementRendererHelper(this, timer); // timer is a GameTimer instance

// Dispose before turning off XNA rendering in OnNavigatedFrom
uiRenderer.Dispose();

That's it. The constructor automatically hooks up all the proper events and even hooks the timer's Draw event so that it draws the scene for you. This does, of course, assume you've already hooked the Draw event for yourself, otherwise you'll have the Silverlight content rendering first. However if you put this into the default page template in the places described above, you'll be fine.

I've attached the sample to this post which contains the UIElementRendererHelper class and a very small sample app for its usage. This code is all licensed under the Microsoft Public License, so have at it. :) UPDATE: Please see this new post regarding UIElementRendererHelper as it includes a newer version of the code that is better suited for Windows Phone 7.1 SDK RTM.

Pros:

1. Extremely simple in concept and in actual implementation.
2. Enabled high level of composition with native WPF elements.
3. Allowed app to leverage WPF input events.

Cons:

1. Not the most efficient process, given a full CPU swap of the pixels each frame. (The R/B swap I did on the CPU could be moved to a pixel shader prior to getting the data which would reduce a good bit of the work, but you're still moving a lot of data from the GPU into the WriteableBitmap every frame).
2. Limits the app from using render targets without more plumbing work as switching away from the main render target prevents the content from making it to the screen.

These limits may be acceptable for some people. I built a real-time shader editing tool using this system and it worked great for my needs. However since some people have asked for more ways to do this integration that provide more performance and don't require the render targets, I went back to the drawing board a bit. After talking with a coworker for a bit (i.e. taking his core idea and running with it ;-) ), I have come up a new solution for this integration using WPF's HwndHost base type.

The idea of the new solution is simple: create a control subclasses WPF's HwndHost to create a child window to which the XNA Framework will render graphics. I leveraged the GraphicsDeviceService from the WinForms sample (with some modifications) to handle GraphicsDevice management and used the CompositionTarget.Rendering event to drive my draw loop. I also used some concepts from the GraphicsDeviceControl in the WinForms sample so that each of my controls will present to the correct window handle, thus enabling apps to have multiple controls running at the same time and sharing the same GraphicsDevice.

This solution, however, is not without its own set of trade offs. Essentially we're flipping the pro/con list from above:

Pros:

1. Rendering requires no CPU copies as we Present directly to the child window where we want our content. This means we get some great performance even with large windows.
2. Since we're presenting directly to the window, we no longer use render targets which allows the app itself to use them for any purpose.

Cons:

1. Due to the nature of hosting a child window, you cannot composite WPF controls on top of the GraphicsDeviceControl (the exception being other child window content such as menus or dialogs).
2. Because the control is really a child window, mouse input is directed to that window's WndProc which means input handling requires a bit more work and causes some duplication of events. My sample does implement enough of this work to be useful, however there may be some things you want that I didn't tackle.
3. This solution requires a few native method calls and an implementation of a WndProc which means the implementation is quite a bit more complex than my previous solution. The usage is still relatively straight forward, however.

What it comes down to is that these two solutions are both useful in different ways and therefore I recommend you look at them both to see which will meet your needs.

At a high level, use of this new sample is very simple. Add a GraphicsDeviceControl into your XAML layout where you want to render XNA graphics. The GraphicsDeviceControl has a LoadContent event that will fire when the GraphicsDevice has been created and a RenderXna event that fires each frame for you to draw your content. Additionally there are a number of "Hwnd*" events for various aspects of mouse input since I couldn't route it through the normal UIElement mouse events. The control also has methods to capture/release the mouse which makes the cursor invisible and forces the cursor position to reset, which is great for tools where you want to rotate an object and not worry about the mouse leaving the view.

The sample application shows how to leverage these various actions in an app with two GraphicsDeviceControls. The top control draws a constantly spinning cube whose color is based on the sliders in the left panel. The bottom control draws a cube in one of three colors than can be changed with the buttons next to the control (or using the 'R', 'G', or 'B' keyboard keys which are set up as hotkeys for those buttons). Additionally you can use the right mouse button and drag to rotate the bottom cube without capturing the mouse or the left mouse button and drag to rotate the cube with capturing the mouse.

Hopefully this proves useful to some of you. Feel free to leave questions or comments below, but to make this point clear: the implementation of this solution is pretty complex and I cannot provide much support as I simply don't have the time. I encourage everyone to play with it, but if you run into issues please start by checking out MSDN documentation or ask on the App Hub forums so that the community can work with you to solve the issue.

Disclaimer: This code was written by a colleague and myself and is not an official solution for mixing XNA Game Studio and WPF. Use of this sample is at your own risk and I make no guarantees as to the quality or usability of the code. I do not guarantee any level of support for users of this sample. The code in the downloadable sample is licensed under the Microsoft Public License, the terms of which can be found here: https://www.microsoft.com/opensource/licenses.mspx#Ms-PL

WpfHostedXna.zip

The short is that you are no longer allowed to set the Volume or IsMuted properties when the active media source is the Zune media library and any time you switch between library music and title music, you must reset any properties on MediaPlayer you care about. These changes were made to ensure applications didn’t alter the user’s music playback experience in a way that the native Zune media player cannot reverse (there is no UI in the Zune media player for music-specific volume or a mute control).

You can see this change today in the emulator. If you were to run the following code you would see that the Volume and IsMuted are ignored while media from the user’s library is playing:

MediaPlayer.Play(new MediaLibrary().Songs);

MediaPlayer.Volume = .5f;

MediaPlayer.IsMuted = true;

Debug.WriteLine("Volume: {0}, IsMuted: {1}", MediaPlayer.Volume, MediaPlayer.IsMuted);

Output:

Volume: 1, IsMuted: False

While the change does affect certain applications attempting to replace the default Zune media player, for some apps and games that play their own music there is a side effect that may cause issues. Because the user may be listening to music on their device when your game starts, if you are setting the Volume or IsMuted properties for your own title music those properties may be ignored.

For example, a common pattern with music playback was something like this where the code would load the song, set the properties accordingly, and then start song playback:

Song song = Content.Load<Song>("BackgroundMusic");

MediaPlayer.Volume = 0.5f;

MediaPlayer.Play(song);

This worked in RTM because we didn’t care what the active media source was; we would set those properties for whatever was playing. With our change this can have issues if the user is listening to music when you try to set the volume.

Moving forward we always recommend playing your music before setting any of the MediaPlayer properties to ensure they are applied to your game’s music:

Song song = Content.Load<Song>("BackgroundMusic");

MediaPlayer.Play(song);

MediaPlayer.Volume = 0.5f;

If you follow this new pattern, it will ensure your title will continue to work as expected when the update rolls out to users. The good part is that you can title update your game today and it will work on the RTM version of Windows Phone as well as the upcoming update.

Stream stream = storageContainer.CreateFile("MyFile.txt");
StreamWriter writer = new StreamWriter(stream);
writer.Write("Stuff");
writer.Close();
stream.Close();

I cringe at this sight.

IDisposable is an interface that objects leveraging native handles or resources will almost always implement to ensure the native resources are released properly and deterministically. If any of the code in there were to cause an exception, you’d risk not releasing those resources at that point. At best you’d see them released when a garbage collection occurred and at worst they wouldn’t get cleaned up at all.

Now, most people will respond with “but there’s so little going on there, what would throw an exception?” and that’s valid for this example, but a lot of games do a lot more in there. Things like serializing (or deserializing) objects, parsing strings into numbers, and so on. There are lots of places where file IO, state saving/loading, and other Stream manipulation can throw, so it’s always better to be on the side of safety rather than stubbornness.

Anytime a type implements IDisposable, that should be your hint to leverage the ‘using’ statement. In an unfortunate case, the ‘using’ keyword is not only used for including namespaces but also for scoping IDisposable objects so that they are guaranteed to be disposed. To reuse the example above, here’s the correct way to write that:

using (Stream stream = storageContainer.CreateFile("MyFile.txt"))
{
    using (StreamWriter writer = new StreamWriter(stream))
    {
        writer.Write("Stuff");
    }
}

Not only does this help scope your objects to prevent using either the stream or writer objects outside of their respective blocks, but it means that even if the code in the middle of the ‘using’ block throws an exception, the objects created in the ‘using’ statements are guaranteed to be disposed. Basically the compiler takes the above code and turns it into this:

{
    Stream stream = storageContainer.CreateFile("MyFile.txt");
    try
    {
        StreamWriter writer = new StreamWriter(stream);

        try
        {
            writer.Write("Stuff");
        }
        finally
        {
            if (writer != null)
                ((IDisposable)writer).Dispose();
        }
    }
    finally
    {
        if (stream != null)
            ((IDisposable)stream).Dispose();
    }
}

(And yes, those extra curly braces are intended; they are used to scope the ‘stream’ object so it cannot be used outside of the ‘using’ block.)

As you can see this is much safer code than the first snippet because it removes the possibility of not disposing those objects and thus ensures that native resources will be properly released when you expect rather than when the next GC occurs or when your app shuts down.

So please, if you find yourself writing code similar to the first snippet, please stop doing that and start ‘using’ IDisposable objects properly.

One of the gotchas with this is that XNA Game Studio 4.0 Windows games default to using the .NET Framework 4.0 Client Profile. The Client Profile is a slimmed down set of the .NET Framework that contains most of the functionality a shipping product would need. This profile was created so that .NET applications can use a smaller .NET Redistributable and therefore be a smaller total package size for users. However the Client Profile does not contain a lot of the build-time features of the .NET Framework such as MSBuild.

In order to use the build-time content pipeline assemblies and functionality, you must switch your game to use the full .NET Framework. This is a pretty easy set of steps to fix up:

1. Right click the game project and select Properties.
2. Find the Target Framework drop down menu which should say .NET Framework 4.0 Client Profile and change that to just .NET Framework 4.0.
3. Save the properties (which will cause a project reload) and your game now targets the full .NET Framework.

Once you’ve switched the target framework, you should find all of the build-time assemblies of .NET Framework show up in the Add References dialog including the build-time content pipeline assemblies.

Do keep in mind that this doesn’t change anything in terms of the EULA or what you are permitted to redistribute with your Windows game; you should read the EULA for XNA Game Studio before distributing your Windows game to ensure that you are following all of the rules. But for simply finding out how to format your XML or for using the build-time components for some other reason, this should help you get up and running.

Allow me now to elaborate on the Xbox 360 implementation of isolated storage as that is likely the most interesting part of this post. A few high level points can be made to give some details about how isolated storage works on Xbox 360:

• IsolatedStorageFile is analogous to the StorageContainer API. This means you can only have one open at a time and you need to dispose of it in order to commit the changes.
• IsolatedStorageFile is a non-player specific container, so it cannot be used for player specific save files.
• IsolatedStorageFile’s underlying container appears in the Xbox UI as “Storage for [Game Title]”; there is no way to customize this as there is with StorageContainers.
• Given that GetUserStoreForApplication() isn’t as asynchronous method, IsolatedStorageFile will not prompt the user for a storage device. It will always attempt to create the container on the same device as the game. This has a few notable results:
  • No UI popups; your game simply has access to the data.
  • Your user cannot choose the device on which their data is saved.
  • If the device holding the game is full or close to it, you may not be able to save enough data or even create the container.
  • If the game is moved from one device to another after the container is created, the game will not see the old container unless it is also moved.

So as the title says, “what storage is right for me?”

Generally speaking the recommendation for games on Xbox 360 is to use the StorageDevice/StorageContainer APIs. These are what professional titles for XBLA and retail will use (or the native equivalents) and will therefore make your game feel more at home in the ecosystem. Isolated storage is there if it fits your needs and your desired experience, but be mindful of the limitations that come with that API as you work on your game.

SaveAsPng and SaveAsJpeg both do as they say. You pass in the Stream to which the data should be written along with the size you want to save the image (useful for saving thumbnails of images by passing in values smaller than the actual texture).

Texture2D.FromStream supports decoding PNG, JPEG, and GIF data from a Stream object. There are two overloads of this method:

Texture2D.FromStream(GraphicsDevice graphicsDevice, Stream stream)

Texture2D.FromStream(GraphicsDevice graphicsDevice, Stream stream,
                     int width, int height, bool zoom)

The first overload should be pretty easy to understand as it simply decodes the image data from the stream and gives you back a Texture2D object.

The second overload takes in a maximum size along with a “zoom” value. If zoom is false, the resulting image will be scaled along its longer dimension to fit in the width and height. If zoom is true, the resulting image is scaled along its shorter dimension to fit in the width and height, cropping as necessary.

As an example, here’s the classic image of Shawn’s cat, found in many of our samples:

This image is 153x248. If we load the image in with Texture2D.FromStream passing in 100 for both the width and height and false for the zoom parameter, this is the resulting texture:

This texture is 61x100. The longest dimension (the height) was scaled to our maximum height and aspect ratio was maintained.

Here’s what we see if we pass in true for the zoom parameter:

This texture is 100x100. The shortest dimension (the width) was scaled to our maximum width and the image was cropped to fit in our 100x100 requested size.

A couple notes about Texture2D.FromStream:

• The Texture2D you receive is always using SurfaceFormat.Color
• The method does not handle color keys, premultiplied alpha, DXT compression, or mipmaps.

Texture2D.FromStream is useful for games that want to decode gamer pictures (which are now a Stream property instead of a Texture2D), album art (on Windows Phone), or user library pictures (on Windows Phone). It’s also nice to pair with the SaveAs* methods for saving and loading screenshots for games on any platform.

Tiny fun fact: the images you see above weren’t generated by me using a paint program; I actually loaded the texture with FromStream and saved out the images with SaveAsPng. So these are really the results you’d get. :)

One aspect of this “engine” was an entity system I was building. The idea was the composition pattern that many people have moved to for game development. At its core it means that an object in my game is nothing more than a container that is then stuffed with a bunch of components that actually build it up. The idea is that you can make building blocks (components) that you then combine to make more interesting objects. It’s a great theory but I’ve never really used a component system for a full game.

The first week was well underway, but I didn’t feel like I was moving fast enough to actually finish what I wanted. Eventually it hit me that the entity system (or at least my mindset) was causing friction between me and productivity. I was spending too much time managing component classes and interfacing between them, and not enough time progressing on the actual gameplay.

Towards the end of the first week, I decided to make a radical move and scrap the entire entity system for a “traditional” class hierarchy approach. Boy was that a good decision! It took me no time at all to build back to where I was and then it was much easier for me to continue to build upon my objects to finish off the game. I’m certain if I had kept fighting the component system, I never would have reached the point I did.

What it boils down to is that if you’re familiar with one paradigm or have a piece of code that works for your needs, stop trying to improve upon it until you have downtime or a real reason to improve it. Your customers won’t care if you used an entity system or class hierarchy, but if you spend all your time obsessing over details like that you might never get to the point of having customers.

The first piece of code was to grab my EasyConfig library, which is a simple three-class library for parsing INI files. I used these INI files to define how each gun behaved, what it looked like, and what sounds it made. The files contained things like damage done, rounds per clip, max ammo, and, for the grenade, the damage falloff exponent so the damage decreased over time. While this code is code I’ve written, I mention it here because it is publicly available to anyone that wants to quickly drop in support for INI files.

The next piece I grabbed was the BoxCollider library from Fabio Policarpo’s FPS demo (available as a download on this page: https://fabio.policarpo.nom.br/Demos.html). BoxCollider is a great little library for handling collisions between boxes, rays, and triangulated meshes. I used the library for the character movement collision as well as to handle raycasting for the guns and the collision portion of my grenade physics (I handled the dynamics myself and just did bouncing; the grenades actually never rotate or roll). I did do a few passes over the code base to optimize some of the methods and remove a few parts that created garbage, but it’s still a very valuable library without those changes.

I also needed to make my grenades go boom, so I snagged the Particle 3D sample and dropped that in. With minor changes just to the size of the particles, the built in explosion effect quickly became the look for my grenades. Is it realistic? No, but then again a giant fireball is way cooler to watch than a bunch of tiny pieces of metal flying around.

What’s the moral here? If you’re trying to actually finish a game, I recommend taking a bit to search around for pieces of code you can leverage to make your development quicker and smoother.