Be Willing To Be Wrong

Early in our work designing Ribbon content, we had little data to go on in
terms of how different content layouts within the Ribbon would affect the usability of
the features being laid out.  Being a new control, there wasn't any direct
information we could fall back on to tell us how to use it most effectively.  So, we did what we almost always do
in the absence of good information: use our gut feeling to make a decision and then get the prototype into the
usability lab as soon as we could to start validating the design.

This was one case in which I was pretty sure I was right.  My instinct
and experience told me that the chunks within a Ribbon tab should be laid out
generally left-to-right in decreasing priority.  (Except of course in
right-to-left languages, in which we would do the opposite.)   To me, this
made perfect sense: people read left-to-right, we've watched people scan menus
and toolbars for years left-to-right.  I just knew logically this would
turn out to be the right decision, so we laid out the content that way in the
Ribbon and headed to the lab.

(Excel "Sheet" tab in recent builds - click to view full picture)

To my surprise, some of the tabs using these left-to-right layouts were
performing terribly.  We would put the key commands over to the left, and
people just never found them, or they wouldn't find them until they scanned the
rest
of the tab first.

In short, my gut instinct and my intelligent guess were both as wrong as they
could have been.

As we watched more people use the prototypes, we started to
understand more the scanning process that was taking place.  Later on, we
did eye-tracking studies to watch how people scanned the Ribbon and it further confirmed our theories based on the earlier tests. 
It turned out that the truth wasn't a
big "A-HA!" moment... it was a series of realizations of how different
combinations of layouts, iconography, text labels, and position affect the usability of a
layout.  In short, the truth is complicated and we needed to try to
understand the relationships at work between layout, scanning pattern, and
usability success.

One important thing we learned, for instance, is that many people tend to
start scanning directly under the name of tab they clicked, proceed to scan around that area
(both directions), and
then continue to the right.  This means that we need to consider where a
tab is positioned in the overall tab set as we consider the layout of that tab.  (The fact
that we ship Office in 30+ localized languages makes this a non-exact science.)

Another thing we noticed was that, totally contrary to my expectation, the
leftmost chunk in a tab is kind of dead space.  It's often the last thing
people scan.  So we switched our thinking relative to this position and
started moving commands that were consistent across the apps as much as possible
into that space.  This means that you spend more time becoming peripherally
aware of the same feature being in the same position and as a result you
counteract the fact that you don't scan it as soon.

It's not even that straightforward.  Certain commands, such as starting a
slide show, or creating a new slide, seem to have a high affinity and
expectation for being on the left and do just fine in that area.  The exact
kinds of layouts present in a tab (such as presence of in-ribbon galleries) also affect the
usability of different relative positions, and we continue to learn more every week.

I was reading a blog entry of someone who was kind of critical and dismissive
about what we're doing
and our designs.  One of his criticisms was "how bad the usability of the
Ribbon would be because it's got icons scattered all over of various sizes." 
What we've learned is actually the opposite.  People can scan disparate
patterns more easily than homogenous patterns.  When we use more
toolbar-like layouts--a bunch of equally-spaced, equally-sized buttons, people
scan them less quickly than when each chunk has a memorable layout.  So we
actually try explicitly to vary the layouts between chunks--it helps people find
the thing they're looking for more quickly.  That's something we wouldn't
have known if we didn't have a commitment to watch people work.

It's becoming more clear every day that there's a nascent language at work
here, and we need to keep working to become fluent in it.  Given that
interaction design is just a form of communication, I guess that shouldn't be a surprise.

There's just no substitute for watching real people try to get work done in
your product.  A well-designed usability methodology, along with an
openness to the possibility that your initial assumptions were wrong, can help
turn a design from a clunker to a winner.