{ End Bracket }

Don’t Be Afraid To Throw Away Your Work

Josh Phillips

We talk often about the "manycore shift"—how it affects us and what we're doing about it—but we frequently gloss over how we can harness it to develop richer client experiences. Exactly what do you do with 4 cores, 64 cores, or 1,000 cores?

For certain domains, it just makes sense: their algorithms are more compelling when they're faster. For others, it takes a bit more thinking. Yes, intuitive user interfacing, such as natural language queries and gestures, can contribute significantly to richer client experiences. But these technologies are research-heavy and difficult to implement. So how else can you make your client apps richer today? Aside from increasing UI responsiveness in general, my radical suggestion is to start throwing away work.

What? Am I actually suggesting that you toss out your hard-earned CPU time as if it were yesterday's news? Yes, I am. In the days of faster and faster single-cores, CPU time was coveted—wasting it was getting in the way of doing something more important. With multicore and especially manycore machines, abundant processing resources afford the opportunity to make more compelling apps by doing both the important stuff and the not-so-important-but-still-useful stuff. You can now compute speculatively: predict what your user will do next, pre-compute the results, and have them ready when they're needed, if they're needed.

Consider a typical photo-editing suite bundled with filters for making your picture look like a painting or a pencil sketch. When you decide to apply a filter, you get a window with some canned picture that's already had each of the filters applied to it. Scroll through this box and search for the filter you want; after you've clicked it, the app chugs along, revealing the transformation to you piecemeal. After the wait, you decide that it doesn't look quite right. You undo, return to the filter box, and repeat.

Let's take this same scenario again and throw some speculative computation on it, sprinkled with 128 cores. Now as soon as the filter window pops up, it has already applied all of the visible filters to a thumbnail of your image. When you scroll to the right, all of those filters have previews as well. What's more, it's also started transforming the original-size image with all of the filters that are currently visible. Now when you click on a filter, the image is already transformed. Once a filter has been selected, the application could also speculatively process the image with different parameters. Take it further and imagine plugging in your camera: as you're browsing through your vacation photos, your computer is already cropping, removing red-eye, and auto adjusting. If you don't like what the computer has done, just say no, throw away the work, and revert to the unedited photo.

Speculative computing isn't limited to photo apps. When you hop into your car, your 4-core navigation system could determine which destination you're most likely to choose, based on the day of the week, time, trip history, and a Bluetooth connection to your phone's calendar and address book. Every time it determines a likely destination, it will pre-compute the route, incorporating traffic data, and, detecting that you're low on fuel, keep the nearest and cheapest gas station in mind. If it turns out you're just making a quick run to the local deli, it can simply throw the work away.

The takeaway here is that we now can stop acting like each cycle of CPU time is worth pure gold. We'll soon have such an abundance of processing power that we'll be more than willing to waste a little for a more compelling app.

And there's no need to wait for the 64-core processors. With quad-cores on shelves now and 8-cores just around the corner, you can start throwing away work today. The Parallel Computing Initiative at Microsoft is working to make multicore and manycore programming easier by abstracting away the difficult threading model. A Community Technology Preview (CTP) of the Parallel Extensions to the Microsoft .NET Framework is now available and is well suited for speculative computing, including task parallelism, data parallelism, coordination, and cancellation APIs. The Parallel Extensions (as well as similar technologies for the native stack) and tools are coming in the next version of Visual Studio.

So speculate. Make suggestions for your users and eliminate the bulk of their mundane tasks. Make your UIs snappy, immersive, and innovative. And don't be afraid to toss out some processing time if you've done it in the name of a great experience.

Josh Phillips is a Program Manager on the Microsoft Parallel Computing Platform Team. He is currently working on libraries, languages, and tools to enable developers to productively utilize concurrency and build the next generation of rich and compelling applications.