Special Windows 10 issue 2015

Volume 30 Number 11

Notifications - Adaptive and Interactive Notifications in Windows 10

By Thomas Fennel | Windows 2015

Few tools in a developer’s toolbox impact user engagement like notifications. They permeate the experience within an OS, whether Windows mobile, Windows desktop, Xbox or even HoloLens. There’s an undeniable lure notifications provide to users. From helping complete a task such as replying to a message, to providing simple and timely information such as news headlines and even delighting them with an experience they didn’t expect—such as a Cortana reminder—notifications provide value to users and engage them with your application.

What Are Notifications in Windows?

Notifications represent a broad category of user engagements throughout an OS. From a user’s perspective, notifications are represented in many experiences. 

Tiles are the most iconic form of notifications for Windows OSes. You see tiles on the Start screen both on Windows desktop and mobile and in various incarnations across other form factors and device types. Tiles provide several benefits to users.

First, they’re a quick way to launch apps using primary tiles—these are tiles you pin from the applications list directly to Start by touching and holding or right-clicking on an app in the list. They also serve as a way to get directly to content deeper within an app using secondary tiles—these are tiles a developer pins programmatically with user consent from within the app. 

Arguably more important is the ability of tiles to offer engagement with customers by providing content from within the app on the tile itself, via live tiles. These give you a way to reach a user without them actually launching your app—the canonical examples being weather forecasts and sports scores for your favorite teams. Getting that information quickly and in one place on the Start menu can delight users.

Toasts are the notifications that pop up at the top of a mobile device or the bottom-right of a desktop screen. They’re interruptive notifications that seek to have users engage with an app by launching it. Before Windows 10, tapping a toast notification could only launch users into the app with some static arguments the developer could set when creating the notification. Also, before Windows 10, there were mobile notifications called alarms and reminders that created larger, modal dialogs to which the user had to respond before doing anything else. These were similar to toasts, and they’ve been folded into a category of notifications called Action Required Toasts.

Badges are the final type of notification. You see badges predominantly on the lock screen, where they’re usually a count of missed items in an application. Or sometimes they’re small glyphs that represent status—think of an exclamation point or something similar. Badges are also represented on primary and secondary tiles.

If It Ain’t Broke, Don’t Fix It

When we started working on adaptive and interactive notifications, we were feverishly driven by the feedback we heard from the developer community about the state of notifications both on Windows Phone and Windows 8. We didn’t want to create something new just because ... we’ve all lived through that in the past and it’s how we ended up with such big differences between the notification technologies, infrastructures and developer models between Windows Phone and Windows desktop. Instead, for Windows 10, we focused on what developers liked about the existing notifications framework in both platforms and made a commitment to doing what was needed, rather than simply trying to completely break free from our legacy. It was an interesting time because not only were we starting Windows 10, we were also integrating the Windows Phone and Windows teams into one platform team. During this transition, it wasn’t always easy to take such a principled approach, but we knew it was righteous to bring the best of both worlds together, and the result—adaptive and interactive notifications—represents the “One Microsoft” approach.

The Tile and Toast Template Catalogs Wouldn’t Scale

So just how do developers design these tile and toast notifications? Previous versions of Windows had a relatively inflexible series of templates that we put into the tile and toast template catalogs. These templates allowed developers to easily and cheaply create notifications that are visually consistent with the Windows design language. However, in order to create all possible permutations of images and text configurations that developers might want, we ended up with hundreds of nearly identical templates.

Figure 1 is a sampling of the tile catalog we offered in Windows Phone 8.1 and Windows 8.1. Text01 is a single line of wrapping text. Text02 is four separate lines of non-wrapping text. Text03 is a header with a single line of wrapping text. Text04 is a header with three lines of non-wrapping text. These clearly won’t scale to every possible combination that a developer might want.

Four Templates from the Tile Template Catalog That Provide Only Subtle Differences in Style and Layout
Figure 1 Four Templates from the Tile Template Catalog That Provide Only Subtle Differences in Style and Layout

For example, what if you need two lines of wrapping text? Unfortunately, that isn’t part of the catalog, so you’re told to use an image-only template and render a custom bitmap. But that results in blurry text when scaled to different resolutions, much higher data usage when downloaded from the cloud, and a performance impact and unreliability when generating the bitmap in a background task.

Toasts were even more problematic, as there were really only two styles. There were technically eight templates, but they broke down into toast with image or toast without image, as shown in Figure 2. Feedback from developers indicated that not only were the layouts for toasts not flexible enough, they weren’t interactive even in simple ways. Toasts with decent amounts of text in them—say three or four lines—couldn’t be consumed easily on small screens because we only showed the first two lines of text at most. Even basic interactivity such as being able to expand the notification to show more content wasn’t possible with the existing framework.

Two Templates from the Toast Template Catalog

Figure 2 Two Templates from the Toast Template Catalog

Not Every Developer Wants (or Can Afford) to Create a Rich Notification

Templates solve a significant problem, because designing, developing and testing a custom-drawn notification such as a live tile is both challenging and costly. Our notification platform runs on a variety of form factors and screen densities, and we support five different tile sizes and two toast sizes, all which would need to be custom designed, developed and tested. Templates can alleviate this pain because developers who want to create a notification that fits within our template language can benefit from minimal development cost and almost zero test cost, and we take on the burden of making sure they look beautiful across all screens.

Developers don’t have to worry about margins, padding, font sizes or weights; they can create something beautiful and consistent with our language by pasting in some XML and changing some <text> attributes.

At the same time, there’s a powerful need to provide something flexible enough for developers to integrate with their own branding. Developers spend a lot of money and time creating a unique look and building a brand to engage users. Highlighting that brand was a central tenet of our work.

What Does This Mean for Templates?

Templates in Windows 10 effectively serve only one main purpose: to target a specific tile size. In Windows 10 desktop and mobile, we have small, medium and wide tiles. In desktop we also have an additional size for large tiles. We think of these tiles in terms of a logical 4-units-wide-by-4-units-high grid. Small tiles occupy 1x1 units on that grid—little squares, really—hence the small name. Medium tiles occupy 2x2 units on that grid, or medium squares. Sensing a trend? Wide tiles are 4x2 units in that grid and large tiles are the full 4x4 units.

When we were designing adaptive and interactive notifications, we concluded that we could do a lot to collapse the catalog to just the key pivot for size. We ended up with only four templates for tiles—TileSmall, TileMedium, TileWide and TileLarge—and one template for toasts: ToastGeneric. In the future we hope to collapse further and have a TileGeneric template for scenarios where the content is the same between all tile sizes. Then developers can just provide one template for their tiles.

In previous versions of Windows, templates also served to define the animation characteristic for the entire surface. A tile that had animations to show a collection of images cycling through it would have its own template. Similarly, a tile that needed to show a collection of thumbnails all at once that animate and fade would need another template. Now animations are also handled through the new hint-* attributes we’ve introduced, which I’ll discuss later.

Even with all that, we will not be deprecating the old template catalog. If developers upgrade to the new Universal Windows Platform (UWP) in Windows 10 without changing the server or client code for constructing their notifications, the app will continue to work great on Windows 10 mobile and desktop. In fact, some things that previously didn’t work consistently or at all between mobile and desktop now work great. For example, using any of the ToastImage­AndText0* templates on Windows Mobile 10 correctly displays the image in the notification where it previously was omitted. 

Adaptive and Interactive Notifications

In Windows 10 we evolved our notification story to provide a richer developer experience and UX by introducing three main features.

The first is adaptive tiles/toasts, which provide a flexible schema to generate more visually rich notifications and have them adaptively show up across different form factors.

The second is interactive toasts, which provide a way to create toast notifications with simple interactions so users can perform quick actions or inline replies inside the notification without activating the app and requiring users to switch context from what they’re currently doing.

The third is to allow apps to subscribe to events about toast notification history changes, so an app can get notified when its own toast notifications are handled by the system or the user. Essentially this is a way to know what has changed for their app inside Action Center.

Adaptive Live Tile and Adaptive and Interactive Toast Notifications in Action

I’ll use the new adaptive and interactive notifications to demonstrate how to build a typical project that showcases the key aspects of tiles and toasts using a real-world scenario. Specifically, I’ll address the types of new notifications you can use in an e-mail application.

I’ll start with the tile of your e-mail application. In terms of how you create the tile itself, either through the app manifest for your primary tile or through the secondary tile APIs, we haven’t made any significant changes in the Windows 10 release, so I’m not going to dive deep into how you do those operations because they’re well-documented already. Defining your tile and toast layout is done using a semantic XML schema, commonly referred to as a notification payload. Really what I’ll focus on for tiles is how to create a more visually rich tile experience using the new tile templates and corresponding XML payload.

I’ll create an adaptive tile notification that shows the user new e-mail notifications. A nice way to engage users is to give them a count of new messages since the last time they opened the application combined with whom the message was from and a few lines of text from the body of the e-mail. Figure 3 shows how this could look in all four sizes on Windows 10 desktop.

Small, Medium, Wide and Large Tiles on Windows 10 Desktop
Figure 3 Small, Medium, Wide and Large Tiles on Windows 10 Desktop

Figure 4 describes the notification payload that represents the tiles in Figure 3

Figure 4 The XML Payload to Create a Medium-Size Adaptive Tile

<binding template="TileMedium" branding="logo">
      <text hint-style="caption">Matt Hidinger</text>
      <text hint-style="captionsubtle">Photos from our trip</text>
      <text hint-style="captionsubtle">Check out these awesome photos
        I took while in New Zealand!</text>
  <text />
      <text hint-style="caption">Lei Xu</text>
      <text hint-style="captionsubtle">Build 2015 Dinner</text>
      <text hint-style="captionsubtle">Want to go out for dinner after
        Build tonight?</text>

There are a few interesting things to note in Figure 4

First, I’m only showing the payload for the medium tile, seen in the TileMedium binding, because the XML for all four sizes is rather long. In order to reproduce all the tiles you see in Figure 3, you’d need to also use the TileSmall, TileWide and TileLarge templates, albeit with very similar content for each. You can put all these bindings into one notification payload.

Second, there’s more content in the actual notification than is showing on the tile. You can see this represented by the two different sets of <group> and <subgroup> tags. Why? Because this tile could be displayed on multiple screen densities, so I’ve included more information in case it gets displayed on a high-density screen that can show more content. The <group> and <subgroup> tags help the system know what content to keep together semantically so things aren’t cut off in strange places—it’s your way to describe a “unit” of information you’d like to keep together if at all possible. 

The third thing to notice is the text element hanging out by itself in the middle of the payload. That’s actually how you specify a simple blank line, as you can see in the large-tile example.

Finally, the hint-* elements I mentioned earlier can be seen in this payload. These are a large part of the new flexibility we’ve introduced to help you do things such as the simple text styling shown in Figure 5. A wide variety of hints let you specify things such as the opacity of an image, the way a profile photo should be cropped and even how to animate items on a tile for certain targeted scenarios. More information on available hints and other tile details is available on the MSDN Blogs at bit.ly/1NYvsbw.

On the Left, a Medium Tile on a Low-Density Screen Versus a High-Density Screen on the Right
Figure 5 On the Left, a Medium Tile on a Low-Density Screen Versus a High-Density Screen on the Right

To further illustrate the point about information density, Figure 5 is an example of the payload from Figure 4 displayed on a low-density screen versus a high-density screen. I know it seems like a medium tile next to a large tile, but in reality it’s a representation of a medium tile on a low-density screen typically found on a lower-priced phone versus a high-density screen found on a flagship phone.

Now I’ll move on to the second interesting and engaging thing you could do, this time using adaptive and interactive toast notifications. Again, as with the tile APIs, nothing in the way you create the toast notifier and toast object has changed in the toast APIs. Just like tiles, what’s really new here is the payload you can use with those APIs. 

You can create a toast notification to be shown when a new e-mail is received and give users some instant gratification by both giving them several lines of text from the e-mail and a couple of simple quick actions to take on the e-mail itself. Two of the most common things people do with e-mail are to mark an item as read, so they don’t have to pay attention to it again when they see their message list later, and delete the mail immediately. On Windows 10 desktop, that kind of toast notification would look like Figure 6.

An Adaptive and Interactive Toast You Might See Used by an E-mail Application
Figure 6 An Adaptive and Interactive Toast You Might See Used by an E-mail Application

The XML in Figure 7 would generate the notification shown in Figure 6

Figure 7 An Adaptive and Interactive Toast You Might See Used by an E-mail Application

    <binding template="ToastGeneric">
      <text>Andrew Bares</text>
      <text>Ideas for blog posts and the template visualizer.</text>
      <text>Hey guys, I've got some great ideas for the blog and
        some feature ideas for the...</text>
      <image placement="AppLogoOverride" hint-crop="circle" src="AndrewBares.png" />
    <action activationType="background" content="Mark Read" arguments="read" />
    <action activationType="background" content="Delete" arguments="delete" />
  <audio src="ms-winsoundevent:Notification.Mail"/>

As with the tile notification payload I showed earlier, there are some interesting new things to notice in the toast payload from Figure 7, as well.

First, unlike the previous templates, you can specify text elements freely and they don’t need their own IDs. I’ve created three lines of text in the toast notification by using three separate text elements.

Second, notice that the first line of text is bold while all subsequent lines are not. For now, you can’t use all the hint-* styles available for text on tile notifications in toasts, so this text styling will always apply for toasts until a future release where we enable more hints and styles on toasts, as well. However, you can use some of the hints. For example, notice that the hint-crop attribute does work here and I’ve cropped the image of Andrew to be a nice circle by using it in combination with the AppLogoOverride placement attribute that lets me display an image in place of where the app logo would normally appear. More information on available hints and other toast notification details is available on the MSDN Blogs at bit.ly/1N3o7GY.

Finally, in the actions section of the payload, notice how there are two buttons created by using individual action elements for the common tasks people perform with e-mails. The arguments are what the app will receive when the buttons are clicked and the application is invoked. In this case, I use an activationType of background because I want to handle these actions in the application’s background task. Alternatively, you can have an activationType of foreground if you want to launch the app to complete the action or an activationType of protocol if you want to invoke a Web site or an app-to-app communication through a standard protocol launch.

Now that I’ve created the XML payload needed to display the notification, I need to handle the actions taken by the user on this interactive toast notification. 

To start, because I’ve chosen an activationType of background, I’ll need a background task in which I can execute code. That task needs to be declared in the apps Package.appxmanifest, as shown in Figure 8.

Declare a Background Task to Handle Interactive Toast Actions in Package.appxmanifest
Figure 8 Declare a Background Task to Handle Interactive Toast Actions in Package.appxmanifest

Then, once the task is registered in the Package.appxmanifest, you can add the code to your background task to actually deal with user actions. In Figure 9 you can see the simple stub where you would add your own code to handle the actions.

Figure 9 Handle Arguments or User Input in a Background Task

namespace Tasks
  public sealed class ToastHandlerTask : IBackgroundTask
    public void Run(IBackgroundTaskInstance taskInstance)
      // Retrieve and consume the pre-defined arguments and user inputs here.
      var details = taskInstance.TriggerDetails as NotificationActionTriggerDetails;
      var arguments = details.Arguments;
      // Handle either marking the mail as read or deleting it from the database.

Now you have an app that can dynamically display visually appealing content on your tile and engage users quickly for their most commonly desired actions with a toast. Your app is more useful and delightful to users, and they can quickly get new information and interact with your experience without ever needing to context-switch into your app from what they’re currently doing. We look forward to the amazing things you’ll do and sincerely hope you’ll get better user engagement through adaptive and interactive notifications in Windows 10.

Thomas Fennel is a principal program manager lead at Microsoft in the Windows Developer Ecosystem and Platform division. Reach him at tfennel@microsoft.com.

Thanks to the following Microsoft technical experts for reviewing this article: Andrew Bares, Matt Hidinger and Lei Xu
Andrew Bares is a program manager specializing in windows application development at Microsoft.

Lei Xu is a program manager II responsible for the toast and action center platform at Microsoft.

Matt Hidinger is a senior program manager responsible for the tile notification platform at Microsoft.