Writing Error Messages for Security Features

 

Everett McKay
Microsoft Corporation

May 17, 2002

Summary: Offers essential information for writing, presenting, and testing security-related messages.

Contents

A Typical Security Message
Information Disclosure
Informed Consent
Progressive Disclosure
Be Specific
Consider Not Asking the Question
Usability Test Your Security Messages
Additional Resources

Good error messages give a notification that a problem occurred, an explanation of why the problem occurred, and a solution so that the user can fix the problem. Good error message text is specific, user-centered, clear, consistent, and courteous. Writing good error messages is hard work.

If you have received an error message that's related to a security feature, chances are good that you found it confusing, it didn't really help you understand the security problem, and you had no idea how to respond correctly. A good question to ask is: Why are security-related error messages so often bad?

By "error messages" I really mean all classes of message boxes, including warnings, confirmations, questions, and status. Much of this information applies to log file entries as well. This article explores the challenge of writing messages for security-related features. I will explain the difficulties in designing good security message text and the information that's required for a good security message, and I'll give some tips for designing and presenting security-related messages.

A Typical Security Message

Let's look at a typical example of a bad security confirmation message.

Figure 1. An example of a bad error message

This message is a notification and has something resembling an explanation. The user can proceed to view the page by clicking Yes, or avoid some vague security risk by clicking No.

So why is this message bad? The message asks a question that the user cannot possibly answer intelligently. The user has requested that Microsoft® Internet Explorer display a page, and this message implicitly advises against it through the wording of the text and by highlighting No as the default choice. The specific security risk that the page poses is not sufficiently explained, so the downside of continuing is totally unclear. In short, this message is bad because it doesn't give the user enough information to make a good decision. Consequently, the message fails to be useful.

Information Disclosure

In general, you want to make error messages as specific and helpful as you can. For security features, specific and helpful information sometimes has an alternative description: information disclosure. Information disclosure occurs when private information is exposed to users who aren't supposed to see it. It is one of the six main security threats to avoid when designing secure software.

If you pay close attention to error messages, you might have a hard time believing that some of them are too helpful. Let's work through a basic example.

Suppose you enter the wrong password when logging on to your computer. Even though Windows could determine exactly what is wrong with the password, giving such specific information would disclose information about the password. Because passwords are secure only if kept secret, they should never be revealed or described in any way. Consequently, rather than giving specific information about what is wrong with the password, Windows gives the following message.

Figure 2. An example of a good error message

This message is a good model of how to give a helpful error message, even when dealing with sensitive data. It presents:

• A notification that a problem occurred (an incorrect password).
• An explanation about why the problem occurred (by implicitly stating that the password was typed incorrectly).
• A solution so that the user can fix the problem (by retyping the password, paying special attention to case sensitivity).

A good security message can give additional helpful information to the user as long as it doesn't reveal anything private. It's fair game to disclose general information about Microsoft® Windows®, the application giving the message, or common user mistakes. In this case, the message reminds the user of the common mistake of typing the password by using the wrong case, such as when the CAPS LOCK key is pressed.

It is also acceptable to give information that can be readily obtained from other sources, such as through documentation or trivial experimentation. Consequently, documented facts such as the permissions or privileges that are required to perform a task are safe. If the user doesn't have permission to perform a task, the fact that he cannot perform the task reveals this information, so this lack of permission can be explained in an error message without jeopardizing security.

When necessary, a security message can disclose private information on a strict "need to know" basis. Microsoft® Internet Information Services (IIS) used to display syntax errors with an error page that shows the problem as well as an excerpt of the offending source code to all users. Such error messages are best described as hacker friendly. A much better approach is to give this specific information only to those who need to know it (in this case the application developer) and give a generic error message to all other users. IIS now uses this approach.

Informed Consent

We can't blame all bad security messages on trying to prevent information disclosure. Consider this gem.

Figure 3. An error message with too much information

Aside from a default choice of No, this message gives the user no clue as to what to do. In fact, it isn't even clear what the user is being asked to do. As with the first example, this message is asking a question that the user cannot possibly answer intelligently. There is plenty of data here, but what does it all mean? Based on the information presented in this message, why would the user choose Yes? Why would the user choose No?

If a message is going to ask the user a security-related question, at the very least it must give the user enough information to make an intelligent decision. This principle is often referred to as informed consent. To make an informed choice about a security issue, the user needs enough information to answer the following questions:

• What is this message really asking me to do? How does it relate to the task I am trying to perform?
• Is the security issue significant or minor?
• If I select the secure choice, what will I not be able to do?
• If I select the insecure choice, what is the worst that can happen? What is likely to happen?
• If I answer incorrectly, can I fix the problem later? If so, how?
• What is the choice recommended by the program? Why?

Resolving a security question without informed consent has no value. Most users know little about security; this can be true even for system administrators in all but the largest organizations. When writing security messages, don't assume that the user is a security expert unless your program is clearly targeted at security experts.

Here is an improved version of the Root Certificate Store message that helps the user answer most of the questions.

Figure 4. Improved version of error message that explains consequences

Sure, this is a large message box, but it now clearly explains the question being asked, the security consequences of the action, and what exactly will happen as the result of the decision. There is no benefit to showing anything less.

Progressive Disclosure

A problem with informed consent is that it usually requires presenting a lot of information to the user—often too much, in fact. The last example presents the minimum amount of information required, but it still lacks crucial information. Specifically, there is no information on how to validate the certificate, and all the gory details that were presented in the original message are now lost.

The best way to present all the information without overwhelming the user is to use progressive disclosure. The base message should have the essential information required for the user to answer the error message question intelligently. Any supplemental information that the user might need should be available on demand, such as through hyperlinks or as a Details, More Information, or Help button.

Here is a version that uses progressive disclosure to help the user validate the certificate.

Figure 5. An error message that uses progressive disclosure

Be Specific

Most messages can be improved by making them more specific. This is certainly true for security messages. Let's take another look at my first example of a bad security-related confirmation message.

Figure 6. An example of a bad error message

As mentioned previously, this message is bad in part because it is completely vague about the security risk at hand. Let's fix this message by making it more specific.

Figure 7. An error message with specific information

This version describes a specific example of the most common security risk that the page might encounter, a hyperlink to get more information about the risks, and specific advice about how to answer the question.

Yes, it's more text. Yes, some users aren't going to read it. But the text isn't excessive, and the essential information that the user needs to answer the question is offset in bold text to make it easy to scan. If the user is unfamiliar with the concept of the question, the hyperlink is available. Most importantly, the user can clearly understand what the security risk is (disclosure of sensitive information) and has a simple criterion upon which to base an intelligent decision. The question is now worth asking.

Security messages can't always be expressed in three simple sentences. Being concise is an important goal, but for security messages it cannot be the primary goal.

The principle to remember is that security issues already fluster users, and most of the security messages that users are likely to see are variations on the question "We have found a security issue. Do you want to proceed securely with impaired functionality or do you want to get your work done?" Users are going to want to get their work done unless presented with an excellent reason not to. Vague statements are rarely motivating, and they totally undermine the value in asking the security question in the first place. Be as specific as you can, without revealing private information.

Consider Not Asking the Question

In his book Secrets and Lies, Bruce Schneier argues that it is futile to expect users to intelligently answer security questions. Here are some of his arguments:

• People are often the weakest link in the security chain.
• People don't understand risks, especially for improbable events.
• Users find security measures intrusive and will gladly work around them as necessary.
• Security warnings don't work because users typically dismiss them without even reading them.
• People just want to get their jobs done.
• Users just don't make good security decisions. The success of the ILOVEYOU and Anna Kournikova worms are clear evidence of this.

This is a strong argument for avoiding security-related questions in the first place. But what are the alternatives? One obvious approach is to not ask the question. This is the preferred approach when you are all but certain that you know the right thing for the user to do.

For example, if a user removes a certificate by using the Content tab of the Internet Explorer Internet Options dialog box, a confirmation message could ask if he also wants to remove the associated private key. The Windows security team at Microsoft realized that deleting the private key is always the right thing to do, so they decided not to bother asking. That's one less bad security message.

Another approach is to use a high-level security policy instead of asking individual low-level security questions. This is the approach used on the Security tab of the Internet Explorer Internet Options dialog box.

Figure 8. A message that uses a high-level security policy

This approach works well because users understand their goals (such as Safe browsing and still functional) far better than they understand security details. Focusing on the user's goals is an important principle that should be applied to all security messages. Furthermore, having a high-level security policy in place requires less interaction from the user, so the user is less likely to make bad decisions or consciously try to bypass security measures. This goal-based approach could be implemented by other applications as well.

Usability Test Your Security Messages

When your team decides that a security message is necessary, be sure to usability test it with your target users early in the development cycle. This step is extremely important because the users' notions of security are often quite unpredictable. Here are some things to check for:

• Did the users understand the context for the message?
• Did they understand the message text?
• Did they understand the security risk?
• Did they obtain all the information required to respond intelligently? Was the information helpful or confusing?
• Did they bother to check any supplemental information?
• What decision did they make? Why?
• Are they confident that they made the right decision?
• Did they understand the consequences of the decision?
• Was the decision correct under the circumstances?

When designing security messages, be sure to supply enough information so that users can respond intelligently, but make sure you aren't revealing any secret information in the message. Use progressive disclosure so that you don't overwhelm the user with information. Consider design alternatives that might allow you to eliminate the message entirely. Lastly, be sure to usability test your security messages to make sure you've got it right.

Additional Resources

Secrets and Lies: Digital Security in a Networked World. Information security expert Bruce Schneier explains what everyone in business needs to know about security to survive and be competitive.

Why Johnny Can't Encrypt: A Usability Evaluation of PGP 5.0. In this paper, Alma Whitten and J.D. Tygar explore why security UI design is different from standard UI design.

About the Author

Everett McKay is a user interface program manager on the Windows Server 2003 administrative tools team and has been at Microsoft since April 2000. He is currently focusing on assisting the security team with their user interfaces. Everett holds a bachelor's and a master's degree in electrical engineering and computer science from MIT and is the author of two books: Developing User Interfaces for Microsoft Windows (Microsoft Press) and Debugging Windows Programs (Addison Wesley).