Appendix J: SDL Requirement: Application Verifier

Article
05/22/2012

Application Verifier is a runtime verification tool for unmanaged code. It helps developers quickly find subtle programming errors that can be extremely difficult to identify with typical application testing. Application Verifier makes it easier to create reliable applications by monitoring an application's interaction with the Microsoft® Windows® operating system. It profiles the application’s use of kernel objects, the registry, the file system, and Win32 APIs (heap, handle, locks, and more).

Why is Application Verifier Important?

Application Verifier can help quickly identify security issues related to heap buffer overruns by enabling it when test scenarios are covered. As a result of using it, your organization could avoid having to release security bulletins related to such problems and save you both money and credibility.

Code Required to Run Application Verifier

Application Verifier should be run on all unmanaged code.

Application Verifier is a tool that detects errors in a process (user mode software) while the process is running. Typical findings include heap corruptions (including heap buffer overruns) and incorrect synchronizations and operations. Whenever Application Verifier finds an issue, it goes into debugger mode. Therefore, either the application being verified should run under a user-mode debugger or the system should run under a kernel debugger.

Application Verifier Usage Scenarios

Application Verifier (available in Visual Studio and as a download) cannot be enabled for a running process. You need to make settings as described in this appendix and then start the application. The settings are persistent until explicitly deleted. Therefore, an application always starts with AppVerifier enabled, regardless of how many times you launch it

The scenarios in this appendix showcase the recommended command-line options for quality gates that you should run during all tests (BVTs, stress, unit, and regression) that exercise the code change.

Testing with Application Verifier

The expectation for this scenario is that the application does not break into debugger mode, and that all tests pass with the same pass rate as when run without Application Verifier enabled.

Enable verifier for the application(s) you wish to test using:
```
appverif /verify <MyApp.exe>
```
Note: /verify will enable the base checks: HANDLE_CHECKS, RPC_CHECKS, COM_CHECKS, LOCK_CHECKS, FIRST_CHANCE_EXCEPTION_CHECKS, and FULL_PAGE_HEAP.
If you are testing a dynamic-link library (DLL), you must enable the verifier for the test .exe that is exercising the DLL.
Run ALL your tests exercising the application.
Analyze any debugger break that you encounter. Debugger breaks signify bugs found by the verifier, and you will need to understand and fix them.
When you are finished, delete all settings made with:
```
appverif /n <MyApp.exe>
```

You can debug any issues you find with Application Verifier by reviewing the Verifier Stop codes within the help contents.

Testing with Application Verifier and Fault Injection

The expectation for this scenario is that the application does not break into debugger mode. Not breaking into the debugger mode means there are no errors that need to be addressed.

The pass rate for the tests may decrease significantly because random fault injections are introduced into the normal operation.

Enable verifier and fault injection for the application(s) you wish to test by using the following command-line syntax:
```
appverif /verify <MyApp.exe> /faults
```
Note If you are testing a DLL, you can apply fault injection on a certain DLL instead of on the entire process. The command-line syntax would be:
```
appverif /verify TARGET [/faults [PROBABILITY [TIMEOUT [DLL ...]]]]
```
For example, appverif /verify <mytest.exe> /faults 5 1000 d3d9.dll
Run all your tests exercising the application.
Analyze any debugger break that you encounter. Debugger breaks signify bugs found by the verifier, and you will need to understand and fix them.
When you are finished, delete all settings made with:
```
appverif /n <MyApp.exe>
```

Note that running with and without fault injection exercises different code paths in an application. Therefore, you must run both scenarios to obtain the full benefit of Application Verifier.

Content Disclaimer

This documentation is not an exhaustive reference on the SDL process as practiced at Microsoft. Additional assurance work may be performed by product teams (but not necessarily documented) at their discretion. As a result, this example should not be considered as the exact process that Microsoft follows to secure all products.

This documentation is provided “as-is.” Information and views expressed in this document, including URL and other Internet website references, may change without notice. You bear the risk of using it.

This documentation does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes.

Licensed under Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported