Usability and prototypes

It's been a while since I last posted and the main reason for my silence has been that I've been busy preparing and running a usability study this week which involved the creation of an HTML prototype to mock up the UI of the product we were studying.

The study is investigating the user experience for a portion of the Whidbey product that has yet to be fully implemented. In order to get good feedback about the design from participants we needed to create a prototype of the product. I suggested that we create HTML prototypes with separate web pages for each step that participants needed to complete in the usability study. Each page would contain a screenshot of VS in a particular state (e.g., the solution explorer showing the current project selected) and would have a set of hotspots within which the user could click to move on to the next page in the sequence. With correctly defined pages and hotspots we could simulate different actions that the user could take with VS. For example, to simulate a right click on a project in the solution explorer, we would have one page containing an image of the Visual Studio IDE with the solution explorer open and a hotspot over the appropriate project in the solution explorer. When the user clicks in the hotspot, we show the next page containing an image of the VS IDE with the solution explorer open as well as the context menu for the project that the user clicked on.

It seemed like a good idea but I underestimated how much work it would involve. For example, to accomodate the different ways that a developer could create a new project in Visual Studio required that we create almost 50 separate web pages with links and hotspots relating each of the pages together as appropriate. To create prototypes for each of the four tasks in the study eventually required two of us to spend three full days creating images etc.

The effort was worthwhile though. We're finishing up the study today and we've been able to use the prototypes to get some pretty useful feedback. It's reasonably early on in the lifecycle of this feature that we should be in a good position to respond to most of the feedback we've collected. You can tell the prototype has worked when you have to keep reminding participants that they are working with a prototype and that what they are looking at is just a bitmap and so they won't be able to scroll through the code they are examining, let alone change the code.

An interesting side effect that I think arises from the fact that we were using a prototype is that participants were more likely to talk about the interaction flow as opposed to the lower level details of the UI such as button labels, colors, positions etc. I think the nature of the prototype made it more obvious to participants that we were more interested in how the different parts of the product flowed together in the different tasks. So the effort involved in creating the prototype, although large, was definitely worthwhile. Something to keep in mind if you're thinking about using prototypes in one of your own usability studies.

Comments

• Anonymous
  February 28, 2004
  What are your thoughts on paper prototyping ? I'd be especially interested in hearing the impact a system with complex user interaction has on the decision what kind of prototyping is appropriate.
  
  My posts on prototyping are here:
  
  On Prototypes : udidahan.weblogs.us/archives/005373.html
  
  Prototyping is ( should be ) learning:
  http://udidahan.weblogs.us/archives/013272.html
• Anonymous
  March 01, 2004
  My overriding goal for anything that has a complex user interaction would be to get feedback as early as possible so that the interaction can be made less complex for the user. The later usability studies or issues are looked at in a product lifecycle, the more expensive any required changes will be.
  
  If getting feedback early on requires the use of paper prototyping then that is definitely the way to go. It's clear though that paper prototyping should not be the only way of collecting usability data. The type and quality of the data collected will be dependent to some extent on the materials used to collect the data. For example, it's likely that you'll get a lot of useful data regarding the interaction flow from a paper prototype, while you're more likely to get detailed feedback about the design of individual dialogs etc in a flow from a richer fidelity prototype.