Security Briefs

Penetration Testing

James A. Whittaker

Contents

Planning
Types of Penetration Tests
Environment Attacks
Input Attacks
Data and Logic Attacks
Don't Be Deterred

When you hear the term penetration testing, you probably envision a lone genius performing arcane tests against some hapless piece of software. And before the renaissance in penetration testing, that was probably a realistic image. Today penetration testing is performed in a far more methodical manner. This is necessary because the Security Development Lifecycle (SDL), and its front-loaded secure design and development focus, is reducing the number of latent defects, making the task of finding vulnerabilities during testing much more difficult. Software security testing is too important to leave to a small group of virtuosos. It must be teachable, methodical, and repeatable so that it can be applied in a wide variety of circumstances.

This is not to say that penetration testing is a science. All testing has exploratory aspects to it; testers apply input, which changes data inside the software causing it to react in any number of ways. The complexity is too great to accurately predict; however, there is much that can be done to plan ahead, and this column will describe how we do that planning at Microsoft.

Planning

With traditional testing, specifications, user documentation, use cases, and other design documentation are abundantly available. This information can be used to design a set of test cases that will verify the specified functionality. But the available sources of information on planning are severely limited. Penetration testing is not about verifying functionality; it's about verifying the absence of insecure functionality. Unfortunately, no one has defined any software development artifacts for these behaviors, and testers are forced to fend for themselves.

The first place to harvest penetration testing information is in the interfaces between the software and its external environment. User interfaces, network interfaces, APIs, and any other place where input is processed are clear attack vectors for hackers. If any of these interfaces are poorly designed or implemented, they may allow maliciously crafted input to enter and wreak havoc. Identifying and documenting these interfaces is a good place to start penetration testing.

The second area requiring special attention is error messages and user alert dialogs, which communicate information from the software to external users. Since such users may have malicious intent, it is important to understand what information is revealed to them and how that information is communicated.

Finally, penetration testers often define disaster scenarios that specify what a successful attack might look like. These misuse cases (or abuse cases if you prefer) often spring from a threat model or from prior known attacks.

Collecting information from all three of these sources is crucial prep work for a penetration test and will help guide you through the actual testing.

Types of Penetration Tests

Testing is about variation—finding the things in the software and its environment that can be varied, varying them, and seeing how the software responds. The goal is to ensure that the software performs reliably and securely under reasonable and even unreasonable production scenarios. So the most fundamental planning a tester can do is to understand what can be varied and what ways that variation needs to be staged for testing.

From a security standpoint, the environment, user input, and internal data and logic are the primary places where such variation can reveal security issues. The environment consists of the files, applications, system resources, and other local or network resources used by the application. Any of these could be the entry point of attack. User input is the data that originates with external (usually untrusted) entities that is parsed and used by the software. Internal data and logic are the internally stored variables and logic paths, which have any number of potential enumerations.

By varying the information in the software's environment, input domain and data/logic paths, you can perform attacks. I'll explore each of these three categories in more detail.

Environment Attacks

Software does not execute in isolation. It relies on any number of binaries and code-equivalent modules, such as scripts and plug-ins. It may also use configuration information from the registry or file system as well as databases and services that may reside anywhere. Each of these environmental interactions may be the source of a security breach and therefore must be tested.

There are also a number of important questions you must ask about the degree of trust that your application has in these interactions, including the following: How much does the application trust its local environment and remote resources? Does the application put sensitive information in a resource (for instance, the registry) that can be read by other applications? Does it trust every file or library it loads without verifying the contents? Can an attacker exploit this trust to force the application to do his bidding?

In addition to the trust questions, penetration testers should watch for DLLs that might be faulty or have been replaced (or modified) by an attacker, binaries, or files with which the application interacts that are not fully protected by access control lists (ACLs) or are otherwise unprotected. Testers must also be on the lookout for other applications that access shared memory resources or store sensitive data in the registry or in temporary files. Finally, testers must consider factors that create system stress, such as a slow network, low memory, and so forth, and determine the impact of these factors on security features.

Environment attacks are often conducted by rigging an insecure environment and then executing the application within that environment to see how it responds. This is an indirect form of testing; the attacks are waged against the environment in which the application is operating. Now let's look at direct testing.

Input Attacks

In penetration testing, the subset of inputs that come from untrusted sources are the most important. These include communication paths such as network protocols and sockets, exposed remote functionality such as DCOM, remote procedure calls (RPCs) and Web services, data files (binary or text), temporary files created during execution, and control files such as scripts and XML, all of which are subject to tampering. Finally, UI controls allowing direct user input, including logon screens, Web front ends, and the like, must also be checked.

Specifically, you'll want to determine whether input is properly controlled: are good inputs allowed in and bad ones (such as long strings, malformed packets, and so forth) kept out? Suitable input checking and file parsing are critical.

You'll need to test to see whether dangerous input can be entered into UI controls, and find out what happens when it is. This includes special characters, encoded input, script fragments, format strings, escape sequences, and so forth. You'll need to determine whether long strings that are embedded in packets fields or in files and are capable of causing memory overflow will get through. Corrupt packets in protocol streams are also a concern. You must watch for crashes and hangs and check the stack for exploitable memory corruption. Finally, you must ensure that such things as validation and error messages happen in the right place (client-side rather than server side) as a proper defense against bad input.

Input attacks really are like lobbing grenades against an application. Some of them will be properly parried and some will cause the software to explode. It's up to the penetration team to determine which are which and initiate appropriate fixes.

Data and Logic Attacks

Some faults are embedded in an application's internal data storage mechanisms and algorithm logic. In such cases, there seem to be design and coding errors where the developer was assuming either a benevolent user or failed to consider some code paths where a user might tread.

Denial of service is the primary example of this category but certainly not the most dangerous. Denial of service attacks can be successful when developers have failed to plan for a large number of users (or connections, files, or whatever inputs cause some resource to be taxed to its limit). However, there are far more insidious logical defects that need to be tested. For example, information disclosure can happen when inputs that drive error messages and other generated outputs reveal exploitable information to an attacker. One practical example of such data that you should always remove is any hardcoded test accounts or test APIs (which are often included in internal builds to aid test automation). These can provide easy access to an attacker. Two more tests you should run are to input false credentials to determine if the internal authorization mechanisms are robust, and choose inputs that vary the code paths. Often one code path is secure but the same functionality can be accessed in a different way, which could inadvertently bypass some crucial check.

Don't Be Deterred

Penetration testing is very different from traditional functional testing; not only do penetration testers lack appropriate documentation, but they also must be able to think like users who intend to do harm. This point is very important—developers often operate under the assumption that no reasonable user would execute a particular scenario, and therefore decline a bug fix. But you really can't take chances like that. Hackers will go to great lengths to find vulnerabilities and no trick, cheat, or off-the-wall test case is out of bounds. The same must be true for penetration testers as well.

Send your questions and comments to briefs@microsoft.com.

James A. Whittaker is a Principal Architect for Visual Studio Team System and author of the How to Break Software series.