Chapter 15 — Measuring .NET Application Performance
Retired Content |
---|
This content is outdated and is no longer being maintained. It is provided as a courtesy for individuals who are still using these technologies. This page may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist. |
Improving .NET Application Performance and Scalability
J.D. Meier, Srinath Vasireddy, Ashish Babbar, Rico Mariani, and Alex Mackman
Microsoft Corporation
May 2004
Related Links
Home Page for Improving .NET Application Performance and Scalability
Send feedback to Scale@microsoft.com
Summary: This chapter provides coding techniques and tools to measure system and application performance. The chapter also includes detailed "What to Measure" and "How to Measure" sections for each of the .NET technologies.
Contents
Objectives
Overview
How to Use This Chapter
Goals of Measuring
Metrics
How Measuring Applies to Life Cycle
Tools and Techniques
Instrumentation
System Resources
.NET Framework Technologies
CLR and Managed Code
ASP.NET
Web Services
Enterprise Services
Remoting
Interop
ADO.NET/Data Access
Summary
Additional Resources
Objectives
- Instrument your applications.
- Measure system resource utilization.
- Measure CLR and managed code performance.
- Measure ASP.NET application and Web service performance.
- Measure Enterprise Services performance.
- Measure .NET remoting performance.
- Measure interop performance.
- Measure ADO.NET and data access performance.
Overview
To determine whether your application meets its performance objectives and to help identify bottlenecks, you need to measure your application's performance and collect metrics. Metrics of particular interest tend to be response time, throughput, and resource utilization (how much CPU, memory, disk I/O, and network bandwidth your application consumes while performing its tasks).
This chapter begins by explaining what measuring is, what the goals of measuring are, and how measuring relates to your application development life cycle. The chapter then presents the tools and technique that you can use to measure performance and obtain metrics. These range from using the inbuilt system performance counters to using custom application instrumentation code. The chapter then shows you the performance counters that you can use to measure system resource utilization. Finally, the remainder of the chapter is divided into technology-focused sections that show you what to measure and how to measure it for each of the core Microsoft® .NET technologies including ASP.NET, Web services, Enterprise Services, .NET remoting, interoperability, and ADO.NET data access.
How to Use This Chapter
To get the most from this chapter, do the following:
- Jump to topics or read from beginning to end. The main headings in this chapter help you locate the topics that interest you. Alternatively, you can read the chapter from beginning to end to gain a thorough appreciation of the issues around measuring .NET application performance.
- Read How Tos. This chapter references several How Tos that are in the "How To" section of this guide. The following How Tos help you use a number of the tools discussed in the chapter and implement a number of the techniques and recommendations:
- "How To: Use CLR Profiler"
- "How To: Monitor the ASP.NET Thread Pool Using Custom Counters"
- "How To: Time Managed Code Using QueryPerformanceCounter and QueryPerformanceFrequency"
- "How To: Use Custom Performance Counters from ASP.NET"
- "How To: Use EIF"
- "How To: Use ACT to Test Performance and Scalability"
- "How To: Perform Capacity Planning for .NET Framework Applications"
- Read Chapter 16, "Testing .NET Application Performance" A key part of the testing process is to capture metrics so that you can determine how close your application is to meeting its performance objectives. The testing chapter presents performance testing processes for load, stress, and capacity testing. Use the current chapter in conjunction with the testing chapter to identify which metrics you should be capturing when performing your tests.
- Download the tools. This chapter discusses Enterprise Instrumentation Framework (EIF) as a way to instrument your applications. EIF is available as a free download from the Microsoft Download Center at https://www.microsoft.com/downloads/details.aspx?FamilyId=80DF04BC-267D-4919-8BB4-1F84B7EB1368&displaylang=en.
Goals of Measuring
Measuring enables you to identify how the performance of your application stands in relation to your defined performance goals and helps you to identify the bottlenecks that affect your application performance. It helps you identify whether your application is moving toward or away from your performance goals. Defining what you will measure, that is, your metrics, and defining the objectives for each metric is a critical part of your testing plan.
Performance objectives include the following:
- Response time or latency
- Throughput
- Resource utilization
Response Time or Latency
Response time is the amount of time taken to respond to a request. You can measure response time at the server or client as follows:
- Latency measured at the server. This is the time taken by the server to complete the execution of a request. This does not take into account the client-to-server latency, which includes additional time for the request and response to cross the network.
- Latency measured at the client. The latency measured at the client includes the request queue, the time taken by the server to complete the execution of the request, and the network latency. You can measure this latency in various ways. Two common approaches are to measure the time taken by the first byte to reach the client (time to first byte, or TTFB) or the time taken by the last byte of the response to reach the client (time to last byte, or TTLB). Generally, you should test this using various network bandwidths between the client and the server.
By measuring latency, you can gauge whether your application takes too long to respond to client requests.
Throughput
Throughput is the number of requests that can be successfully served by your application per unit time. It can vary depending on the load (number of users) applied to the server. Throughput is usually measured in terms of requests per second. In some systems, throughput may go down when there are many concurrent users. In other systems, throughput remains constant under pressure but latency begins to suffer, perhaps due to queuing. Other systems have some balance between maximum throughput and overall latency under stress.
Resource Utilization
You identify resource utilization costs in terms of server and network resources. The primary resources are the following:
- CPU
- Memory
- Disk I/O
- Network I/O
You can identify the resource cost on a per-operation basis. Operations might include browsing a product catalog, adding items to a shopping cart, or placing an order. You can measure resource costs for a given user load or you can average resource costs when the application is tested using a given workload profile.
Workload Profile
A workload profile consists of an aggregate mix of users performing various operations. For example, for a load of 200 concurrent users, the profile might indicate that 20 percent of users perform order placement, 30 percent add items to a shopping cart, while 50 percent browse the product catalog. This helps you identify and optimize areas that consume an unusually large proportion of server resources.
Metrics
Metrics provide information about how close your application is to your performance goals. In addition, they also help you identify problem areas and bottlenecks within your application. You can group various metric types under the following categories:
- Network. Network metrics are related to network bandwidth usage.
- System. System metrics are related to processor, memory, disk I/O, and network I/O.
- Platform. Platform metrics are related to ASP.NET, and the .NET common language runtime (CLR).
- Application. Application metrics include custom performance counters.
- Servicelevel. Service level metrics are related to your application, such as orders per second and searches per second.
How Measuring Applies to Life Cycle
You should start to measure as soon as you have a defined set of performance objectives for your application. This should be early in the application design phase. The process of continual measuring is shown in Figure 15.1.
Figure 15.1: Measuring in relation to the application life cycle
You must continue to measure application performance throughout the life cycle to determine whether your application is trending toward or away from its performance objectives.
Tools and Techniques
Measuring involves collecting data during the various stages of your application's development life cycle. You might need to collect data during prototyping, application development, performance testing, and tuning, and in a production environment. The following tools and techniques help you to collect data:
- System and platform metrics
- Network monitoring tools
- Profiling tools
- Tools for analyzing log files
- Application instrumentation
System and Platform Metrics
You can use various tools to collect system and platform metrics:
- System Monitor. This is a standard component of the Windows operating system. You can use it to monitor performance objects and counters and instances of various hardware and software components.
- Microsoft Operations Manager (MOM). You can install MOM agents on individual servers that collect data and send it a centralized MOM server. The data is stored in the MOM database, which can be a dedicated SQL Server or the Microsoft SQL Server 2000 Desktop Engine (MSDE) version of Microsoft SQL Server. MOM is suitable for collecting large amounts of data over a long period of time.
- Stress tools such as Application Center Test (ACT). You can use tools such as ACT to simulate clients and collect data during the duration of a test.
Network Monitoring Tools
You can monitor network performance with the following tools:
- Internet Protocol Security (IPSec) monitor. You can use IPSec monitor to confirm whether your secured communications are successful. In this way you can monitor any possible pattern of security-related or authentication-related failures.
- Network Monitor (NetMon). You use NetMon to monitor the network traffic. You can use it to capture packets sent between client and server computers. It provides valuable timing information as well as packet size, network utilization, addressing, and routing information and many other statistics that you can use to analyze system performance.
Profiling Tools
There are a variety of tools available that allow you to profile .NET applications and SQL Server:
CLR Profiler. This allows you to create a memory allocation profile for your application so you can see how much allocation occurs, where it occurs, and how efficient the garbage collector is within you application. By using the various profile views, you can obtain many useful details about the execution, allocation, and memory consumption of your application.
For more information about using this tool, see "How To: Use CLR Profiler" in the "How To" section of this guide.
SQL Profiler. This profiling tool is installed with SQL Server. You can use it to identify slow and inefficient queries, deadlocks, timeouts, recompilations, errors, and exceptions for any database interactions.
SQL Query Analyzer. This tool is also installed with SQL Server. You can use it to analyze the execution plans for SQL queries and stored procedures. This is mostly used in conjunction with the SQL Profiler.
Third-party tools. There are various third-party tools that you can use to profile .NET applications including VTune from Intel and Xtremesoft Appmetrics. These tools help you identify and tune your application bottlenecks.
Tools for Analyzing Log Files
Analyzing log files is an important activity when tuning or troubleshooting your live application. Log files can provide usage statistics for various modules of the application, a profile of the users accessing your application, errors and exceptions, together with other diagnostics that can help identify performance issues and their possible cause.
You can analyze various logs including Internet Information Services (IIS) logs, SQL Server logs, Windows event logs, and custom application logs. You can use various third-party tools to analyze and extract details from these log files.
Application Instrumentation
In addition to the preceding tools, you can instrument your code for capturing application-specific information. This form of information is much more fine-grained than that provided by standard system performance counters. It is also a great way to capture metrics around specific application scenarios.
For example, instrumentation enables you to measure how long it takes to add an item to a shopping cart, or how long it takes to validate a credit card number. There are a number of ways that you can instrument your code. These are summarized in the next section, and each approach is expanded upon in subsequent sections.
Instrumentation
Instrumentation is the process of adding code to your application to generate events to allow you to monitor application health and performance. The events are generally logged to an appropriate event source and can be monitored by suitable monitoring tools such as MOM. An event is a notification of some action.
Instrumentation allows you to perform various tasks:
- Profile applications. Profiling enables you to identify how long a particular method or operation takes to run and how efficient it is in terms of CPU and memory resource usage.
- Collect custom data. This might include custom performance counters that you use to monitor application-specific activity, such as how long it takes to place an order.
- Trace code. This allows you to understand the application code path and all the methods run for a particular use case.
What Options Are Available?
You can use various tools and technologies to help you instrument your application. The right choice depends on your development platform, logging frequency, the volume of data being logged, and how you plan to monitor. You have several options:
- Event Tracing for Windows (ETW)
- Window Management Instrumentation (WMI)
- Custom performance counters
- Enterprise Instrumentation Framework (EIF)
- Trace and Debug classes
When Do You Use Each Option?
The various options available for logging are suitable for different scenarios. The following questions help you make an informed choice:
What do you want to accomplish with instrumentation?
There are various goals for instrumentation. For example, if you need only debugging and tracing features, you might use these features mostly in a test environment where performance is not an issue. But if you plan to log the business actions performed by each user, this is very important from a performance point of view and needs to be considered at design time.
If you need to trace and log activity, you need to opt for a tool that lets you specify various levels of instrumentation through a configuration file.
How frequently do you need to log events?
Frequency of logging is one of the most important factors that helps you decide the right choice of the instrumentation tool for you. For example, the event log is suitable only for very low-frequency events, whereas a custom log file or ETW is more suitable for high-frequency logging. The custom performance counters are best for long-term trending such as week-long stress tests.
Is your instrumentation configurable?
In a real-life scenario for a typical application, you might want to instrument it so the data collected is helpful in various application stages, such as development (debugging/tracing), system testing, tuning, and capacity planning.
All the code stubs for generating the data need to be inserted in the application during the development stage of the life cycle. The code stubs for generating such a huge amount of data may themselves add to performance overhead. However, in most scenarios, it is a small subset of data that you are interested in during a particular stage of the life cycle. You need configurable instrumentation so the relevant set can be turned on at will to minimize performance overhead.
Based on the preceding considerations, the usage scenarios for the various options are as follows.
Event Tracing for Windows (ETW)
ETW is suitable for logging high-frequency events such as errors, warnings or audits. The frequency of logging can be in hundred of thousands of events each second in a running application. This is ideal for server applications that log the business actions performed by the users. The amount of logging done may be high, depending on the number of concurrent users connected to the server.
Windows Management Instrumentation (WMI)
Windows Management Instrumentation is the core management technology built into the Windows operating system. WMI supports a very wide range of management tools that lets you analyze the data collected for your application.
Logging to a WMI sink is more expensive than other sinks, so you should generally do so only for critical and high-visibility events such as a system driver failure.
Custom Performance Counters
You can use custom counters to time key scenarios within your application. For example, you can use a custom counter to time how long order placement takes or how long it takes to retrieve customer records. For more information about custom counters, see the following How Tos in the "How To" section of this guide:
- "How To: Monitor the ASP.NET Thread Pool Using Custom Counters"
- "How To: Time Managed Code Using QueryPerformanceCounter and QueryPerformanceFrequency"
- "How To: Use Custom Performance Counters from ASP.NET"
Enterprise Instrumentation Framework (EIF)
EIF permits .NET applications to be instrumented to publish a broad spectrum of information such as errors, warnings, audits, diagnostic events, and business-specific events. You can configure which events you generate and where the events go, to the Windows Event Log service or SQL Server, for example. EIF encapsulates the functionality of ETW, the Event Log service, and WMI. EIF is suitable for large enterprise applications where you need various logging levels in your application across the tiers. EIF also provides you a configuration file where you can turn on or turn off the switch for logging of a particular counter.
EIF also has an important feature of request tracing that enables you to trace business processes or application services by following an execution path for an application spanning multiple physical tiers.
More Information
- For more information and a working sample of implementing EIF, see "How To: Use EIF" in the "How To" section of this guide.
- The levels of granularity of tracing are also configurable. EIF is available as a free download from the Microsoft Download Center at the Microsoft Enterprise Instrumentation Framework page at https://www.microsoft.com/downloads/details.aspx?FamilyId=80DF04BC-267D-4919-8BB4-1F84B7EB1368&displaylang=en.
Trace and Debug Classes
The Trace and Debug classes allow you to add functionality in your application mostly for debugging and tracing purposes.
The code added using the Debug class is relevant for debug builds of the application. You can print the debugging information and check logic for assertions for various regular expressions in your code. You can write the debug information by registering a particular listener, as shown in the following code sample.
public static void Main(string[] args){
Debug.Listeners.Add(new TextWriterTraceListener(Console.Out));
Debug.AutoFlush = true;
Debug.WriteLine("Entering Main");
Debug.WriteLine("Exiting Main");
}
The code stubs added using the Trace class are enabled for both the debug and release builds of the application. The Trace class can be used to isolate problems by enabling the trace through the execution code path.
In Visual Studio .NET, tracing is enabled by default. When using the command line build for C# source code, you need to add /d:Trace flag for the compiler or #define TRACE in the source code to enable tracing. For Visual Basic .NET source code you need to add /d:TRACE=True for the command line compiler. The following code sample uses the Trace class for adding the tracing feature in your code.
public static void Main(string[] args){
Trace.Listeners.Add(new TextWriterTraceListener(Console.Out));
Trace.AutoFlush = true;
Trace.WriteLine("Entering Main");
Trace.WriteLine("Exiting Main");
}
More Information
For more information, see Knowledge Base article 815788, "HOW TO: Trace and Debug in Visual C# .NET," at https://support.microsoft.com/default.aspx?scid=kb;en-us;815788.
System Resources
When you need to measure how many system resources your application consumes, you need to pay particular attention to the following:
- Processor. Processor utilization, context switches, interrupts and so on.
- Memory. Amount of available memory, virtual memory, and cache utilization.
- Network. Percent of the available bandwidth being utilized, network bottlenecks.
- Disk I/O. Amount of read and write disk activity. I/O bottlenecks occur if read and write operations begin to queue.
The next sections describe the performance counters that help you measure the preceding metrics.
Processor
To measure processor utilization and context switching, you can use the following counters:
Processor\% Processor Time
Threshold: The general figure for the threshold limit for processors is 85 percent.
Significance: This counter is the primary indicator of processor activity. High values many not necessarily be bad. However, if the other processor-related counters are increasing linearly such as % Privileged Time or ProcessorQueueLength, high CPU utilization may be worth investigating.
Processor\% Privileged Time
Threshold: A figure that is consistently over 75 percent indicates a bottleneck.
Significance: This counter indicates the percentage of time a thread runs in privileged mode. When your application calls operating system functions (for example to perform file or network I/O or to allocate memory), these operating system functions are executed in privileged mode.
Processor\% Interrupt Time
Threshold: Depends on processor.
Significance: This counter indicates the percentage of time the processor spends receiving and servicing hardware interrupts. This value is an indirect indicator of the activity of devices that generate interrupts, such as network adapters. A dramatic increase in this counter indicates potential hardware problems.
System\Processor Queue Length
Threshold: An average value consistently higher than 2 indicates a bottleneck.
Significance: If there are more tasks ready to run than there are processors, threads queue up. The processor queue is the collection of threads that are ready but not able to be executed by the processor because another active thread is currently executing. A sustained or recurring queue of more than two threads is a clear indication of a processor bottleneck. You may get more throughput by reducing parallelism in those cases.
You can use this counter in conjunction with the Processor\% Processor Time counter to determine if your application can benefit from more CPUs. There is a single queue for processor time, even on multiprocessor computers. Therefore, in a multiprocessor computer, divide the Processor Queue Length (PQL) value by the number of processors servicing the workload.
If the CPU is very busy (90 percent and higher utilization) and the PQL average is consistently higher than 2 per processor, you may have a processor bottleneck that could benefit from additional CPUs. Or, you could reduce the number of threads and queue more at the application level. This will cause less context switching, and less context switching is good for reducing CPU load. The common reason for a PQL of 2 or higher with low CPU utilization is that requests for processor time arrive randomly and threads demand irregular amounts of time from the processor. This means that the processor is not a bottleneck but that it is your threading logic that needs to be improved.
System\Context Switches/sec
Threshold: As a general rule, context switching rates of less than 5,000 per second per processor are not worth worrying about. If context switching rates exceed 15,000 per second per processor, then there is a constraint.
Significance: Context switching happens when a higher priority thread preempts a lower priority thread that is currently running or when a high priority thread blocks. High levels of context switching can occur when many threads share the same priority level. This often indicates that there are too many threads competing for the processors on the system. If you do not see much processor utilization and you see very low levels of context switching, it could indicate that threads are blocked.
Memory
To measure memory utilization and the impact of paging, you can use the following counters:
Memory\Available Mbytes
Threshold: A consistent value of less than 20 to 25 percent of installed RAM is an indication of insufficient memory.
Significance: This indicates the amount of physical memory available to processes running on the computer. Note that this counter displays the last observed value only. It is not an average.
Memory\Page Reads/sec
Threshold: Sustained values of more than five indicate a large number of page faults for read requests.
Significance: This counter indicates that the working set of your process is too large for the physical memory and that it is paging to disk. It shows the number of read operations, without regard to the number of pages retrieved in each operation. Higher values indicate a memory bottleneck.
If a low rate of page-read operations coincides with high values for Physical Disk\% Disk Time and Physical Disk\Avg. Disk Queue Length*,* there could be a disk bottleneck. If an increase in queue length is not accompanied by a decrease in the pages-read rate, a memory shortage exists.
Memory\Pages/sec
Threshold: Sustained values higher than 500 indicate a bottleneck.
Significance: This counter indicates the rate at which pages are read from or written to disk to resolve hard page faults. Multiply the values of the Physical Disk\Avg. Disk sec/Transfer and Memory\Pages/sec counters. If the product of these counters exceeds 0.1, paging is taking more than 10 percent of disk access time, which indicates that you need more RAM.
Memory\Pool Nonpaged Bytes
Threshold: Watch the value of Memory\Pool Nonpaged Bytes for an increase of 10 percent or more from its value at system startup.
Significance: If there is an increase of 10 percent or more from its value at startup, a serious leak is potentially developing.
Server\Pool Nonpaged Failures
Threshold: Regular nonzero values indicate a bottleneck.
Significance: This counter indicates the number of times allocations from the nonpaged pool have failed. It indicates that the computer's physical memory is too small. The nonpaged pool contains pages from a process's virtual address space that are not to be swapped out to the page file on disk, such as a process' kernel object table. The availability of the nonpaged pool determines how many processes, threads, and other such objects can be created. When allocations from the nonpaged pool fail, this can be due to a memory leak in a process, particularly if processor usage has not increased accordingly.
Server\Pool Paged Failures
Threshold: No specific value.
Significance: This counter indicates the number of times allocations from the paged pool have failed. This counter indicates that the computer's physical memory or page file is too small.
Server\Pool Nonpaged Peak
Threshold: No specific value.
Significance: This is the maximum number of bytes in the nonpaged pool that the server has had in use at any one point. It indicates how much physical memory the computer should have. Because the nonpaged pool must be resident, and because there has to be some memory left over for other operations, you might quadruple it to get the actual physical memory you should have for the system.
Memory\Cache Bytes
Threshold: No specific value.
Significance: Monitors the size of cache under different load conditions. This counter displays the size of the static files cache. By default, this counter uses approximately 50 percent of available memory, but decreases if the available memory shrinks, which affects system performance.
Memory\Cache Faults/sec
Threshold: No specific value.
Significance: This counter indicates how often the operating system looks for data in the file system cache but fails to find it. This value should be as low as possible. The cache is independent of data location but is heavily dependent on data density within the set of pages. A high rate of cache faults can indicate insufficient memory or could also denote poorly localized data.
Cache\MDL Read Hits %
Threshold: The higher this value, the better the performance of the file system cache. Values should preferably be as close to 100 percent as possible.
Significance: This counter provides the percentage of Memory Descriptor List (MDL) Read requests to the file system cache, where the cache returns the object directly rather than requiring a read from the hard disk.
Disk I/O
To measure disk I/O activity, you can use the following counters:
PhysicalDisk\Avg. Disk Queue Length
Threshold: Should not be higher than the number of spindles plus two.
Significance: This counter indicates the average number of both read and writes requests that were queued for the selected disk during the sample interval.
PhysicalDisk\Avg. Disk Read Queue Length
Threshold: Should be less than two.
Significance: This counter indicates the average number of read requests that were queued for the selected disk during the sample interval.
PhysicalDisk\Avg. Disk Write Queue Length
Threshold: Should be less than two.
Significance: This counter indicates the average number of write requests that were queued for the selected disk during the sample interval.
PhysicalDisk\Avg. Disk sec/Read
Threshold: No specific value.
Significance: This counter indicates the average time, in seconds, of a read of data from the disk.
PhysicalDisk\Avg. Disk sec/Transfer
Threshold: Should not be more than 18 milliseconds.
Significance: This counter indicates the time, in seconds, of the average disk transfer. This may indicate a large amount of disk fragmentation, slow disks, or disk failures. Multiply the values of the Physical Disk\Avg. Disk sec/Transfer and Memory\Pages/sec counters. If the product of these counters exceeds 0.1, paging is taking more than 10 percent of disk access time, so you need more RAM.
PhysicalDisk\Disk Writes/sec
Threshold: Depends on manufacturer's specification.
Significance: This counter indicates the rate of write operations on the disk.
Network I/O
To measure network I/O, you can use the following counters:
Network Interface\Bytes Total/sec
Threshold: Sustained values of more than 80 percent of network bandwidth.
Significance: This counter indicates the rate at which bytes are sent and received over each network adapter. This counter helps you know whether the traffic at your network adapter is saturated and if you need to add another network adapter. How quickly you can identify a problem depends on the type of network you have as well as whether you share bandwidth with other applications.
Network Interface\Bytes Received/sec
Threshold: No specific value.
Significance: This counter indicates the rate at which bytes are received over each network adapter. You can calculate the rate of incoming data as a part of total bandwidth. This will help you know that you need to optimize on the incoming data from the client or that you need to add another network adapter to handle the incoming traffic.
Network Interface\Bytes Sent/sec
Threshold: No specific value.
Significance: This counter indicates the rate at which bytes are sent over each network adapter. You can calculate the rate of incoming data as a part of total bandwidth. This will help you know that you need to optimize on the data being sent to the client or you need to add another network adapter to handle the outbound traffic.
Server\Bytes Total/sec
Threshold: Value should not be more than 50 percent of network capacity.
Significance: This counter indicates the number of bytes sent and received over the network. Higher values indicate network bandwidth as the bottleneck. If the sum of Bytes Total/sec for all servers is roughly equal to the maximum transfer rates of your network, you may need to segment the network.
Protocol related counters:
Protocol_Object\Segments Received/sec
Protocol_Object\Segments Sent/sec
Protocol_Object can be TCP, UDP, NetBEUI, NWLink IPX, NWLink NetBIOS, NWLink SPX, or other protocol layer performance objects.
Threshold: Application-specific.
Significance: Protocol-related counters help you narrow down the traffic to various protocols because you might be using one or more protocols in your network. You may want to identify which protocol is consuming the network bandwidth disproportionately.
Processor\% Interrupt Time
Threshold: Depends on processor.
Significance: This counter indicates the percentage of time the processor spends receiving and servicing hardware interrupts. This value is an indirect indicator of the activity of devices that generate interrupts, such as network adapters.
.NET Framework Technologies
The .NET Framework provides a series of performance counters, which you can monitor using System Monitor and other monitoring tools. To measure other aspects of .NET application performance, you need to add instrumentation to your applications. Subsequent sections in this chapter explain what you need to measure for each of .NET technology starting with the CLR and managed code in general and then how to measure it.
CLR and Managed Code
This section describes what you need to measure in relation to the CLR and managed code and how you capture the key metrics. This applies to all managed code, regardless of the type of assembly, for example, ASP.NET application, Web service, serviced component, and data access component.
What to Measure
When measuring the processes running under CLR some of the key points to look for are as follows:
- Memory. Measure managed and unmanaged memory consumption.
- Working set. Measure the overall size of your application's working set.
- Exceptions. Measure the effect of exceptions on performance.
- Contention. Measure the effect of contention on performance.
- Threading. Measure the efficiency of threading operations.
- Code access security. Measure the effect of code access security checks on performance.
How to Measure
You can measure the performance of your managed code by using system performance counters. The main counters used to measure managed code performance and to identify CLR related bottlenecks are summarized in Table 15.1.
Table 15.1: Performance Counters Used to Measure Managed Code Performance
Area | Counter |
---|---|
Memory | Process\Private Bytes
.NET CLR Memory\% Time in GC .NET CLR Memory\# Bytes in all Heaps .NET CLR Memory\# Gen 0 Collections .NET CLR Memory\# Gen 1 Collections .NET CLR Memory\# Gen 2 Collections .NET CLR Memory\# of Pinned Objects .NET CLR Memory\Large Object Heap Size |
Working Set | Process\Working Set |
Exceptions | .NET CLR Exceptions\# of Exceps Thrown / sec |
Contention | .NET CLR LocksAndThreads\Contention Rate/sec
.NET CLR LocksAndThreads\Current Queue Length |
Threading | .NET CLR LocksAndThreads\# of current physical Threads
Thread\% Processor Time Thread\Context Switches/sec Thread\Thread State |
Code Access Security | .NET CLR Security\Total RunTime Checks
.NET CLR Security\Stack Walk Depth |
Memory
To measure memory consumption, use the following counters:
Process\Private Bytes
Threshold: The threshold depends on your application and on settings in the Machine config file. The default for ASP.NET is 60 percent available physical RAM or 800 MB, whichever is the minimum. Note that .NET Framework 1.1 supports 1,800 MB as the upper bound instead of 800 MB if you add a /3GB switch in your Boot.ini file. This is because the .NET Framework is able to support 3 GB virtual address space instead of the 2 GB for the earlier versions.
Significance: This counter indicates the current number of bytes allocated to this process that cannot be shared with other processes. This counter is used for identifying memory leaks.
.NET CLR Memory\% Time in GC
Threshold: This counter should average about 5 percent for most applications when the CPU is 70 percent busy, with occasional peaks. As the CPU load increases, so does the percentage of time spent performing garbage collection. Keep this in mind when you measure the CPU.
Significance: This counter indicates the percentage of elapsed time spent performing a garbage collection since the last garbage collection cycle. The most common cause of a high value is making too many allocations, which may be the case if you are allocating on a per-request basis for ASP.NET applications. You need to study the allocation profile for your application if this counter shows a higher value.
.NET CLR Memory\# Bytes in all Heaps
Threshold: No specific value.
Significance: This counter is the sum of four other counters — Gen 0 Heap Size, Gen 1 Heap Size, Gen 2 Heap Size, and Large Object Heap Size. The value of this counter will always be less than the value of Process\Private Bytes, which also includes the native memory allocated for the process by the operating system. Private Bytes - # Bytes in all Heaps is the number of bytes allocated for unmanaged objects.
This counter reflects the memory usage by managed resources.
.NET CLR Memory\# Gen 0 Collections
Threshold: No specific value.
Significance: This counter indicates the number of times the generation 0 objects are garbage-collected from the start of the application. Objects that survive the collection are promoted to Generation 1. You can observe the memory allocation pattern of your application by plotting the values of this counter over time.
.NET CLR Memory\# Gen 1 Collections
Threshold: One-tenth the value of # Gen 0 Collections.
Significance: This counter indicates the number of times the generation 1 objects are garbage-collected from the start of the application.
.NET CLR Memory\# Gen 2 Collections
Threshold: One-tenth the value of # Gen 1 Collections.
Significance: This counter indicates the number of times the generation 2 objects are garbage-collected from the start of the application. The generation 2 heap is the costliest to maintain for an application. Whenever there is a generation 2 collection, it suspends all the application threads. You should profile the allocation pattern for your application and minimize the objects in generation 2 heap.
.NET CLR Memory\# of Pinned Objects
Threshold: No specific value.
Significance: When .NET-based applications use unmanaged code, these objects are pinned in memory. That is, they cannot move around because the pointers to them would become invalid. These can be measured by this counter. You can also pin objects explicitly in managed code, such as reusable buffers used for I/O calls. Too many pinned objects affect the performance of the garbage collector because they restrict its ability to move objects and organize memory efficiently.
.NET CLR Memory\Large Object Heap Size
Threshold: No specific values.
Significance: The large object heap size shows the amount of memory consumed by objects whose size is greater than 85 KB. If the difference between # Bytes in All Heaps and Large Object Heap Size is small, most of the memory is being used up by large objects. The large object heap cannot be compacted after collection and may become heavily fragmented over a period of time. You should investigate your memory allocation profile if you see large numbers here.
Working Set
To measure the working set, use the following counter:
Process\Working Set
Threshold: No specific value.
Significance: The working set is the set of memory pages currently loaded in RAM. If the system has sufficient memory, it can maintain enough space in the working set so that it does not need to perform the disk operations. However, if there is insufficient memory, the system tries to reduce the working set by taking away the memory from the processes which results in an increase in page faults. When the rate of page faults rises, the system tries to increase the working set of the process. If you observe wide fluctuations in the working set, it might indicate a memory shortage. Higher values in the working set may also be due to multiple assemblies in your application. You can improve the working set by using assemblies shared in the global assembly cache.
Exceptions
To measure exceptions, use the following counter:
.NET CLR Exceptions\# of Exceps Thrown / sec
Threshold: This counter value should be less than 5 percent of Request/sec for the ASP.NET application. If you see more than 1 request in 20 throw an exception, you should pay closer attention to it.
Significance: This counter indicates the total number of exceptions generated per second in managed code. Exceptions are very costly and can severely degrade your application performance. You should investigate your code for application logic that uses exceptions for normal processing behavior. Response.Redirect, Server.Transfer, and Response.End all cause a ThreadAbortException in ASP.NET applications.
Contention
To measure contention, use the following counters:
.NET CLR LocksAndThreads\Contention Rate / sec
Threshold: No specific value.
Significance: This counter displays the rate at which the runtime attempts to acquire a managed lock but without a success. Sustained nonzero values may be a cause of concern. You may want to run dedicated tests for a particular piece of code to identify the contention rate for the particular code path.
.NET CLR LocksAndThreads\Current Queue Length
Threshold: No specific value.
Significance: This counter displays the last recorded number of threads currently waiting to acquire a managed lock in an application. You may want to run dedicated tests for a particular piece of code to identify the average queue length for the particular code path. This helps you identify inefficient synchronization mechanisms.
Threading
To measure threading, use the following counters:
.NETCLR LocksAndThreads\# of current physical Threads
Threshold: No specific value.
Significance: This counter indicates the number of native operating system threads currently owned by the CLR that act as underlying threads for .NET thread objects. This gives you the idea of how many threads are actually spawned by your application.
This counter can be monitored along with System\Context Switches/sec. A high rate of context switches almost certainly means that you have spawned a higher than optimal number of threads for your process. If you want to analyze which threads are causing the context switches, you can analyze the Thread\Context Swtiches/sec counter for all threads in a process and then make a dump of the process stack to identify the actual threads by comparing the thread IDs from the test data with the information available from the dump.
Thread\% Processor Time
Threshold: No specific value.
Significance: This counter gives you the idea as to which thread is actually taking the maximum processor time. If you see idle CPU and low throughput, threads could be waiting or deadlocked. You can take a stack dump of the process and compare the thread IDs from test data with the dump information to identify threads that are waiting or blocked.
Thread\Context Switches/sec
Threshold: No specific value.
Significance: The counter needs to be investigated when the System\Context Switches/sec counter shows a high value. The counter helps in identifying which threads are actually causing the high context switching rates.
Thread\Thread State
Threshold: The counter tells the state of a particular thread at a given instance.
Significance: You need to monitor this counter when you fear that a particular thread is consuming most of the processor resources.
Code Access Security
To measure code access security, use the following counters:
.NET CLR Security\Total RunTime Checks
Threshold: No specific value.
Significance: This counter displays the total number of runtime code access security checks performed since the start of the application. This counter used together with the Stack Walk Depth counter is indicative of the performance penalty that your code incurs for security checks.
.NET CLR Security\Stack Walk Depth
Threshold: No specific value.
Significance: This counter displays the depth of the stack during that last runtime code access security check. This counter is not an average. It just displays the last observed value.
Timing Your Code Path
There are often requirements that you need to know how much time a particular code path takes during execution. You may need this information when you are comparing various prototypes in the design stage or profiling the APIs or critical code paths during the development stage. You need to instrument your code to calculate the time duration and log it in an appropriate event sink such as Event Log or Windows Trace Session Manager. The timing code in your application may look like the following.
QueryPerfCounter myTimer = new QueryPerfCounter();
// Measure without boxing
myTimer.Start();
for(int i = 0; i < iterations; i++)
{
// do some work to time
}
myTimer.Stop();
// Calculate time per iteration in nanoseconds
double result = myTimer.Duration(iterations);
More Information
For more information about the approach and a working sample, see "How To: Time Managed Code Using QueryPerformanceCounter and QueryPerformanceFrequency" in the "How To" section of this guide.
ASP.NET
This section describes what you need to do to measure ASP.NET application performance and how to capture the key metrics. For more information about measuring ASP.NET Web services, see "Web Services" later in this chapter.
What to Measure
To effectively determine ASP.NET performance, you need to measure the following:
- Throughput. This includes the number of requests executed per second and throughput related bottlenecks, such as the number of requests waiting to be executed and the number of requests being rejected.
- Cost of throughput. This includes the cost of processor, memory, disk I/O, and network utilization.
- Queues. This includes the queue levels for the worker process and for each virtual directory hosting a .NET Web application.
- Response time and latency. The response time is measured at the client as the amount of time between the initial request and the response to the client (first byte or last byte). Latency generally includes server execution time and the time taken for the request and response to be sent across the network.
- Cache utilization. This includes the ratio of cache hits to cache misses. It needs to be seen in larger context because the virtual memory utilization may affect the cache performance.
- Errors and exceptions. This includes numbers of errors and exceptions generated.
- Sessions. You need to be able to determine the optimum value for session timeout and the cost of storing session data locally versus remotely. You also need to determine the session size for a single user.
- Loading. This includes the number of assemblies and application domains loaded, and the amount of committed virtual memory consumed by the application.
- View state size. This includes the amount of view state per page.
- Page size. This includes the size of individual pages.
- Page cost. This includes the processing effort required to serve pages.
- Worker process restarts. This includes the number of times the ASP.NET worker process recycles.
How to Measure
You measure ASP.NET performance primarily by using system performance counters. Figure 15.2 shows the main performance counters that you use to measure ASP.NET performance and how they relate to the ASP.NET request processing cycle.
Figure 15.2: The ASP.NET request/response cycle and key performance counters
Table 15.2 summarizes the key performance counters that you use to measure ASP.NET performance and to identify ASP.NET bottlenecks.
Table 15.2: Performance Counters Used to Measure ASP.NET Performance
Area | Counter |
---|---|
Worker Process | ASP.NET\Worker Process Restarts |
Throughput | ASP.NET Applications\Requests/Sec
Web Service\ISAPI Extension Requests/sec ASP.NET\Requests Current ASP.NET Applications\Requests Executing ASP.NET Applications\Requests Timed Out |
Response time / latency | ASP.NET\ Request Execution Time |
Cache | ASP.NET Applications\Cache Total Entries
ASP.NET Applications\Cache Total Hit Ratio ASP.NET Applications\Cache Total Turnover Rate ASP.NET Applications\Cache API Hit Ratio ASP.NET Applications\Cache API Turnover Rate ASP.NET Applications\Output Cache Entries ASP.NET Applications\Output Cache Hit Ratio ASP.NET Applications\Output Cache Turnover Rate |
Throughput
To measure ASP.NET throughput, use the following counters:
ASP.NET Applications\Requests/Sec
Threshold: Depends on your business requirements.
Significance: The throughput of the ASP.NET application on the server. It is one the primary indicators that help you measure the cost of deploying your system at the necessary capacity.
Web Service\ISAPI Extension Requests/sec
Threshold: Depends on your business requirements.
Significance: The rate of ISAPI extension requests that are simultaneously being processed by the Web service. This counter is not affected by the ASP.NET worker process restart count, although the ASP.NET Applications\Requests/Sec counter is.
Cost of Throughput
To measure the throughput cost in terms of the amount of system resources that your requests consume, you need to measure CPU utilization, memory consumption, and the amount of disk and network I/O. This also helps in measuring the cost of the hardware needed to achieve a given level of performance. For more information about how to measure resource costs, see "System Resources" earlier in this chapter.
Requests
To measure ASP.NET requests, use the following counters:
ASP.NET\Requests Current
Threshold: No specific value.
Significance: The number of requests currently handled by the ASP.NET ISAPI. This includes those that are queued, executing, or waiting to be written to the client. ASP.NET begins to reject requests when this counter exceeds the requestQueueLimit defined in the processModel configuration section.
If ASP.NET\Requests Current is greater than zero and no responses have been received from the ASP.NET worker process for a duration greater than the limit specified by <processModel responseDeadlockInterval=/>, the process is terminated and a new process is started.
ASP.NET Applications\Requests Executing
Threshold: No specific value.
Significance: The number of requests currently executing. This counter is incremented when the HttpRuntime begins to process the request and is decremented after the HttpRuntime finishes the request.
For information about a hotfix for this counter, see Knowledge Base article 821156, "INFO: ASP.NET 1.1 June 2003 Hotfix Rollup Package," at https://support.microsoft.com/kb/821156.
ASP.NET Applications\ Requests Timed Out
Threshold: No specific value.
Significance: The number of requests that have timed out. You need to investigate the cause of request timeouts. One possible reason is contention between various threads. A good instrumentation strategy helps capture the problem in the log. To investigate further, you can debug and analyze the process using a run-time debugger such as WinDbg.
Queues
To measure ASP.NET queuing, use the following counters:
ASP.NET\ Requests Queued
Threshold: No specific value.
Significance: The number of requests currently queued. Queued requests indicate a shortage of I/O threads in IIS 5.0. In IIS 6.0, this indicates a shortage of worker threads. Requests are rejected when ASP.NET\Requests Current exceeds the requestQueueLimit (default = 5000) attribute for <processModel> element defined in the Machine.config file. This can happen when the server is under very heavy load.
The queue between IIS and ASP.NET is a named pipe through which the request is sent from one process to the other. In IIS 5.0, this queue is between the IIS process (Inetinfo.exe) and the ASP.NET worker process (Aspnet_wp.exe.) In addition to the worker process queue there are separate queues for each virtual directory (application domain.) When running in IIS 6.0, there is a queue where requests are posted to the managed thread pool from native code. There is also a queue for each virtual directory.
You should investigate the ASP.NET Applications\Requests In Application Queue and ASP.NET\Requests Queued to investigate performance issues.
ASP.NET Applications\ Requests In Application Queue
Threshold: No specific value.
Significance: There is a separate queue that is maintained for each virtual directory. The limit for this queue is defined by the appRequestQueueLimit attribute for <httpRunTime> element in Machine.config. When the queue limit is reached the request is rejected with a "Server too busy" error.
ASP.NET\ Requests Rejected
Threshold: No specific value.
Significance: The number of requests rejected because the request queue was full. ASP.NET worker process starts rejecting requests when ASP.NET\Requests Current exceeds the requestQueueLimit defined in the processModel configuration section. The default value for requestQueueLimit is 5000.
ASP.NET\ Requests Wait Time
Threshold: 1,000 milliseconds. The average request should be close to zero milliseconds waiting in queue.
Significance: The number of milliseconds the most recent request was waiting in the named pipe queue between the IIS and the ASP.NET worker process. This does not include any time spent in the queue for a virtual directory hosting the Web application.
Response Time and Latency
You can measure response time (and latency) from a client and server perspective. From the client perspective, you can measure the time taken for the first byte of the response to reach the client and the time taken for the last time to reach the client. The latency here includes network latency (the time taken for the request and response to travel over the network) and server latency (the time taken for the server to process the request.) You measure time to first byte (TTFB) and time to last byte (TTLB) by using client-side tools such as ACT. On the server-side, you measure the time taken by ASP.NET to process a request by using the ASP.NET\Request Execution Time performance counter.
The key items to measure to determine response time and latency are shown in Figure 15.3.
Figure 15.3: Response time and latency measurements
To measure response time and latency, capture the following metrics:
TTFB
Threshold: Depends on your business requirements.
Significance: This is the time interval between sending a request to the server and receiving the first byte of the response. The value varies depending on network bandwidth and server load. Use client tools such as ACT to obtain this metric.
TTLB-
Threshold: Depends on your business requirements.
Significance: This is the time interval between sending a request to the server and receiving the last byte of the response. Again, the value varies depending upon network bandwidth and server load. Use client tools such as ACT to obtain this metric.
ASP.NET\Request Execution Time
Threshold: The value is based on your business requirements.
Significance: This is the number of milliseconds taken to execute the last request. The execution time begins when the HttpContext for the request is created, and stops before the response is sent to IIS. Assuming that user code does not call HttpResponse.Flush, this implies that execution time stops before sending any bytes to IIS, or to the client.
More information
For more information about measuring ASP.NET response time and latency, see the following resources:
- Knowledge Base article 815161, "HOW TO: Measure ASP.NET Responsiveness with the Web Application Stress Tool," at https://support.microsoft.com/default.aspx?scid=kb;en-us;815161.
- "How To: Use ACT to Test Performance and Scalability" in the "How To" section of this guide.
Cache Utilization
To measure ASP.NET caching, use the following counters:
ASP.NET Applications\ Cache Total Entries
Threshold: No specific value.
Significance: The current number of entries in the cache which includes both user and internal entries. ASP.NET uses the cache to store objects that are expensive to create, including configuration objects and preserved assembly entries.
ASP.NET Applications\ Cache Total Hit Ratio
Threshold: With sufficient RAM, you should normally record a high (greater than 80 percent) cache hit ratio.
Significance: This counter shows the ratio for the total number of internal and user hits on the cache.
ASP.NET Applications\Cache Total Turnover Rate
Threshold: No specific value.
Significance: The number of additions and removals to and from the cache per second (both user and internal.) A high turnover rate indicates that items are being quickly added and removed, which can impact performance.
ASP.NET Applications\Cache API Hit Ratio
Threshold: No specific value.
Significance: Ratio of cache hits to misses of objects called from user code. A low ratio can indicate inefficient use of caching techniques.
ASP.NET Applications\ Cache API Turnover Rate
Threshold: No specific value.
Significance: The number of additions and removals to and from the output cache per second. A high turnover rate indicates that items are being quickly added and removed, which can impact performance.
ASP.NET Applications\ Output Cache Entries
Threshold: No specific value.
Significance: The number of entries in the output cache. You need to measure the ASP.NETApplications\ Output Cache Hit Ratio counter to verify the hit rate to the cache entries. If the hit rate is low, you need to identify the cache entries and reconsider your caching mechanism.
ASP.NET Applications\ Output Cache Hit Ratio
Threshold: No specific value.
Significance: The total hit-to-miss ratio of output cache requests.
ASP.NET Applications\ Output Cache Turnover Rate
Threshold: No specific value.
Significance: The number of additions and removals to the output cache per second. A high turnover rate indicates that items are being quickly added and removed, which can impact performance.
Errors and Exceptions
To measure ASP.NET exceptions, use the following counters:
ASP.NET Applications\ Errors Total/sec
Threshold: No specific value.
Significance: The total number of exceptions generated during preprocessing, parsing, compilation, and run-time processing of a request. A high value can severely affect your application performance. This may render all other results invalid.
ASP.NET Applications\ Errors During Execution
Threshold: No specific value.
Significance: The total number of errors that have occurred during the processing of requests.
ASP.NET Applications\ Errors Unhandled During Execution/sec
Threshold: No specific value.
Significance: The total number of unhandled exceptions per second at run time.
Sessions
To measure session performance, you need to be able to determine the optimum value for session timeout, and the cost of storing session state in process and out of process.
Determining an Optimum Value for Session Timeout
Setting an optimum value for session timeout is important because sessions continue to consume server resources even if the user is no longer browsing the site. If you fail to optimize session timeouts, this can cause increased memory consumption.
To determine the optimum session timeout
Identify the average session length for an average user based on the workload model for your application. For more information about workloads, see "Workload Modeling" in Chapter 16, "Testing .NET Application Performance."
Set the session duration in IIS for your Web site to a value slightly greater than the average session length. The optimum setting is a balance between conserving resources and maintaining the users session.
Identify the metrics you need to monitor during load testing. A sample snapshot of relevant metrics is shown in Table 15.3.
Table 15.3: Metrics
Object Counter Instance ASP.NET Applications Requests/Sec Your virtual dir Processor % Processor Time _Total Memory Available Mbytes N/A Process Private Bytes aspnet_wp Run load tests with the simultaneous number of users set to the value identified for your workload model. The duration of the test should be more than the value configured for the session timeout in IIS.
Repeat the load test with the same number of simultaneous users each time. For each of the iterations, increase the value for the session timeout by a small amount. For each of the iterations, you should observe the time interval where there is an overlap of existing users who have completed their work but who still have an active session on the server and a new set of users who have active sessions increases. You should observe increased memory consumption. As soon as the sessions for the old set of users time out, memory consumption is reduced. The height of the spikes in memory utilization depends on the amount of data being stored in the session store on a per-user basis.
Continue to increase the timeout until the spikes tend to smooth out and stabilize. Set your session state to be as small as possible while still avoiding the memory utilization spikes.
Set a value for the session timeout that is well above this limit.
In Process vs. Remote Session Stores
You might need to use a remote session state store because you want to install your application in a load balanced Web farm. If you do so, you need to do the following:
- Ensure that all objects that are to be stored are serializable.
- Measure the cost of storing sessions on a remote server. You need to consider network latency and the frequency with which your application is likely to access the session state store.
To measure relative in-process vs. remote server processing cost
Identify your performance objectives, such as response time (TTFB/TTLB) and resource utilization levels (CPU, memory, disk I/O and network I/O).
**Note **You should identify performance objectives by using performance modeling during the early phases of your application requirements capture and design. For more information, see Chapter 2, "Performance Modeling."
Configure the session mode for your application to use the in-process state store by using the following setting in Web.config.
<session mode="InProc" ... />
Perform a load test using the workload profile identified for your application. For more information, see "Load Testing Process" in Chapter 16, "Testing .NET Application Performance."
Identify the metrics which you need to monitor during the load tests. A sample snapshot is shown in Table 15.4.
Table 15.4: Snapshot of Load Test Metrics
Object Counter Instance ASP.NET Applications Requests/Sec Your virtual dir ASP.NET Request Execution Time N/A Processor % Processor Time _Total Memory Available Mbytes N/A Process Private Bytes Aspnet_wp You also need to measure the TTLB values on the client used for generating load on the server.
Run the load tests with the simultaneous number of users set to the value identified for your workload model.
Execute the test with the session mode set to "StateServer" in your application's Web.config file.
Compare the values of TTLB and other metrics to determine the cost of storing sessions on a remote server. You should aim to meet the performance objective set for your application response time.
Loading
To measure ASP.NET loading, use the following counters:
.NET CLR Loading\ Current appdomains
Threshold: The value should be same as number of Web applications plus one. The additional one is the default application domain loaded by the ASP.NET worker process.
Significance: The current number of application domains loaded in the process.
.NET CLR Loading\ Current Assemblies
Threshold: No specific value.
Significance: The current number of assemblies loaded in the process. ASP.NET Web pages (.aspx files) and user controls (.ascx files) are "batch compiled" by default, which typically results in one to three assemblies, depending on the number of dependencies. Excessive memory consumption may be caused by an unusually high number of loaded assemblies. You should try to minimize the number of Web pages and user controls without compromising the efficiency of workflow.
Assemblies cannot be unloaded from an application domain. To prevent excessive memory consumption, the application domain is unloaded when the number of recompilations (.aspx, .ascx, .asax) exceeds the limit specified by <compilation numRecompilesBeforeAppRestart=/>.
Note If the <%@ page debug=%> attribute is set to true, or if <compilation debug=/> is set to true, batch compilation is disabled.
.NET CLR Loading\ Bytes in Loader Heap
Threshold: No specific value.
Significance: This counter displays the current size (in bytes) of committed memory across all application domains. Committed memory is the physical memory for which space has been reserved in the paging file on disk.
ViewState Size
View state can constitute a significant portion of your Web page output, particularly if you are using large controls such as the DataGrid or a tree control.
It is important to measure the size of view state because this can have a big impact on the size of the overall response and on the response time. You can measure view state by enabling page level tracing. To enable tracing for a page, add the Trace attribute and set its value to true as shown in the following code.
<%@ Page Language="C#" Trace="True" %>
Page Size
If you need to identify the size of the response sent for a request made for a particular page, one option is to enable logging of bytes sent in the IIS log.
To enable logging of bytes sent in IIS
Open the Internet Information Services management console.
Expand the Web Sites node.
Right-click the Web site you need to enable logging for, and then click Properties.
The Default Web Site Properties dialog box appears.
On the Web Site tab, click Properties.
The Extended Logging Properties dialog box appears.
Select the ExtendedProperties tab and select the ExtendedProperties check box.
Select the Bytes Sent (sc-bytes) check box.
Click OK to close all dialog boxes.
To observe the size of a page
- Use Internet Explorer to browse to the page for which you need to know the page size.
- Use Windows Explorer to browse to the <root drive>\Windows\system32\Logfiles\W3SVC1 folder.
- Open the log file. You should see the sc-bytes column added to the log file. You can observe the value for the request made in step 1.
**Note **Do not enable this on your production server as this significantly increases the log file size.
Page Cost
You can measure the page cost in terms of time taken to serve the page and the processor cycles needed to completely execute the request for the page:
Measuring page cost in terms of CPU cycles. It is possible to calculate the cost of each request execution for a particular page in terms of CPU cycles by using Transaction Cost Analysis (TCA). The formula used to calculate the cost in terms of CPU cycles is as follows:
Cost (Mcycles/request) = ((number of processors x processor speed) x processor use))/ number of requests per second
For detailed information on the TCA methodology, see "How To: Perform Capacity Planning for .NET Applications" in the "How To" section of this guide.
Measuring page cost in terms of time. To measure the time taken for request execution, you can measure the ASP.NET\Request Execution Time counter. Instead of an average value, if you need to calculate the page cost for each request you can add the code that follows to your application's Global.asax file. The code calculates the time by calculating the interval between the Application_BeginRequest and Application_EndRequest events.
**Note **Logging incurs some performance overhead, so avoid logging this information in your production environment.
<%@ import namespace="System.IO" %> <script runat=server> //static members for the writing syncronization private static StreamWriter _writer; private static object _lock = new object(); //change this to a directory that the aspnet account has read\write //permissions to private static string _fileName = string.Format(@"c:\temp\log_{0}.txt",DateTime.Now.ToFileTime()); //member variables for tracking start/end times private DateTime _startTime; private DateTime _endTime; public static StreamWriter GetStaticWriter() { //make sure we're thread safe here... if(_writer==null){ lock(_lock){ if(_writer==null){ _writer = new StreamWriter(_fileName,false); _writer.WriteLine("IP ADDRESS \tSTART TIME \tEND TIME \tDIFF \tURL"); _writer.WriteLine("===============\t============\t============\t================\t========================="); _writer.Flush(); } } } return _writer; } public static void LogText(string str){ GetStaticWriter().WriteLine(str); GetStaticWriter().Flush(); } protected void Application_BeginRequest(Object sender, EventArgs e){ _startTime = DateTime.Now; } protected void Application_EndRequest(Object sender, EventArgs e){ _endTime = DateTime.Now; LogText(string.Format("{0,-12}\t{1:hh:mm:ss.fff}\t{2:hh:mm:ss.fff}\t{3}\t{4}",Request.ServerVariables["REMOTE_ADDRESS"].ToString(),_startTime,_endTime,_endTime-_startTime,Request.Url)); } protected void Application_End(Object sender, EventArgs e){ //release the writer // Even if this doesn't execute, when the appdomain gets shutdown //it will be released anyways if(_writer!=null) _writer.Close(); } </script>
The log file for your application will be similar to the one shown in Figure 15.4.
Figure 15.4: Log file
Worker Process Restarts
To measure worker process restarts, use the following counter:
ASP.NET\ Worker Process Restarts
Threshold: Depends on your business requirements.
Significance: The number of times the Web application recycles and the worker process recycles.
Web Services
This section describes what you need to do to measure ASP.NET Web service performance and how you capture the key metrics.
What to Measure
To effectively determine ASP.NET performance, you need to measure the following:
- Throughput. Measure the number of requests executed per second and throughput-related bottlenecks, such as the number of requests waiting to be executed, and the number of requests being rejected.
- Cost of throughput. Measure processor, memory, disk I/O, and network utilization.
- Queues. Measure the queue levels for the worker process and for each virtual directory hosting a .NET Web service.
- Request Execution Time. Measure the time taken to execute the request at the server.
- Latency or Response Time. Measure the time taken for Web method execution and for the response to be returned to the client.
- Cache utilization. Measure the ratio of cache hits to cache misses. This needs to be seen in larger context because the virtual memory utilization may affect the cache performance.
- Errors and exceptions. Measure the numbers of errors and exceptions generated.
- Sessions. Determine and measure the optimum value for session timeout, and the cost of storing session data locally versus remotely. You also need to determine the session size for a single user.
- XML serialization. Measure the cost of XML serialization.
How to Measure
To measure Web service performance, you can use many of the same counters used to measure ASP.NET application performance. For details about these counters, see "ASP.NET" earlier in this chapter. The other main factor to measure is the impact of XML serialization.
Serialization Cost
Web services use the XmlSerializer class to serialize and de-serialize data. You can calculate the cost of serializing a particular object in terms of memory overhead and the size of data by using the following code snippet.
using System;
using System.IO;
using System.Xml;
using System.Xml.Serialization;
using System.Text;
using System.Data;
//A sample class for serialization
public class MyClass{
public string name;
public string surName;
public MyClass(){
name="FirstName";
surName="LastName";
}
}
class Class1{
private static long startMemory, endMemory, gcMemory, actualMemory,
overHeadMemory;
private static double percentOverhead;
static void Main(string[] args){
//stream to which the class object shall be serialized
Stream fs = new FileStream("SomeFile.txt",FileMode.Create);
MyClass mc = new MyClass();
XmlSerializer xs = new XmlSerializer(typeof(MyClass));
XmlWriter writer = new XmlTextWriter(fs, new UTF8Encoding());
// Clean up the GC memory and measure the measuring as the baseline
before
// performing the serialization
System.GC.Collect();
System.GC.WaitForPendingFinalizers();
startMemory = System.GC.GetTotalMemory(false);
xs.Serialize(writer,mc);
//Calculate the overhead and the amount of data after serialization
CalculateOverhead(fs.Position);
DisplayInfo();
writer.Close();
fs.Close();
Console.ReadLine();
}
public static void CalculateOverhead(long streamPosition){
endMemory = System.GC.GetTotalMemory(false);
gcMemory = endMemory-startMemory;
actualMemory = streamPosition;
overHeadMemory = gcMemory-actualMemory;
percentOverhead = ((double)(overHeadMemory * 100)) /
(double)actualMemory;
}
public static void DisplayInfo() {
Console.WriteLine("Total amount of data after serialization -
>"+actualMemory);
Console.WriteLine("Total memory used by GC for serialization -
>"+gcMemory);
Console.WriteLine("Overhead memory used serialization ->"+overHeadMemory);
Console.WriteLine("Percent overhead ->"+percentOverhead);
}
}
You can use CLR Profiler to get a more complete picture of allocations and the actual objects allocated. For more information, see "How To: Use CLR Profiler" in the "How To" section of this guide.
Enterprise Services
This section describes what you need to do to measure Enterprise Services application performance and how you capture the key metrics.
What to Measure
To measure Enterprise Services performance, you need to capture the following metrics that primarily relate to object usage:
- Number of objects activated
- Number of objects in use
- Time duration for which an object is in call
- Cost of a transaction in terms of resources and time taken to complete the transaction
- Cost in terms of processor and memory utilization for specific method calls to serviced components
- Optimum size for the object pool
During the design, prototyping, or development stage, you may also require to evaluate the performance of components using various features of Enterprise Services, such as components with and without object pooling and components with and without just-in-time activation.
How to Measure
To measure Enterprise Services performance, you can capture metrics displayed by the Component Services administration tool.
Components Services Administration Tool
The Component Services tool enables you to measure metrics for the components hosted in COM+. However, you can only observe the values only at a given instant as there is no logging feature available.
To view Enterprise Services metrics
- Open the Component Services administration tool.
- Expand Component Services, Computers, My Computer, COM+ Applications, and then expand your application in the left hand tree control.
- Click the Components folder.
- On the View menu, click Status.
This enables you to see the metrics in Table 15.5, in the details pane of the console.
Table 15.5: Enterprise Services Metrics
Column | Description |
---|---|
ProgID | Identifies a specific component. |
Objects | Shows the number of objects that are currently held by client references. |
Activated | Shows the number of objects that are currently activated. |
Pooled | Shows the total number of objects created by the pool, including objects that are in use and objects that are deactivated. |
In Call | Identifies objects in call. |
Call Time (ms) | Shows the average call duration (in milliseconds) of method calls made in the last 20 seconds (up to 20 calls). Call time is measured at the object and does not include the time used to traverse the network. |
Optimum Size for the Object Pool
Object pooling is usually enabled for components that take a long time to be initialized. These components tend to hold on some sort of server resources such as a network connection or a file handle. If the pool size is too big, the objects may end up blocking on server resources without doing any useful work. If the pool size is too small, the requests may end up waiting in a queue waiting to get hold of an object to service the request.
You need to execute the load test in the following procedure to identify the optimum size for your pool.
To determine the optimum size for the object pool
Identify the transactions per second (T), response time, and the workload model set during the performance modeling phase for your application.
For a given load of simultaneous users and test duration of Y seconds, you should be able to complete a specific number of transactions in the given time frame without crossing the limit set for the response time and other system resources. The calculation for total transactions uses the following formula:
Total transactions T (in Y seconds) = T transactions/sec * Y sec
Start with a value of the maximum object pool that is lower than the concurrent load of users for the transactions.
Identify the metrics that need to be measured during the load testing of the System Monitor. For example, if you have an ASP.NET client, see Table 15.6.
Table 15.6: Measuring ASP.NET Performance
Object Counter Instance ASP.NET Applications Requests/Sec Your virtual dir ASP.NET Applications Requests In Application Queue Your virtual dir ASP.NET Request Execution Time N/A Processor % Processor Time _Total Memory Available Mbytes N/A Process Private Bytes aspnet_wp
You also need to measure the metrics related to object pool usage from the Component Services console.
To measure object pool usage
- Run the load tests with the simultaneous number of users set to the value identified for your workload model.
- Perform multiple iterations for the load test with same number of simultaneous users. Each of the iterations increases the value for Max Object Pool Size by a small amount. For each of the iterations, there is a change in the values of the metrics.
- Plot a graph for the metrics with respect to the varying object pool size. Decide on the optimum value for the object pool based on the various performance objectives set for the metrics identified.
Third-Party Tools
If you need to profile the components at a more detailed level with respect to time taken, resource cost for a particular transaction, and so on, you need to either instrument your code or use third-party tools such as Xtremesoft AppMetrics.
Remoting
This section describes what you need to do to measure .NET remoting performance and how you capture the key metrics.
What to Measure
To effectively determine .NET remoting performance, you need to measure the following:
- Throughput. Measure the throughput of the remote component.
- Serializationcost and amount of data. Measure the cost of serializing parameters and return values.
- Number of TCP connections. Measure the number of TCP connections established with the remote host.
- Contention for singleton objects. Measure the impact of locking and queuing.
How to Measure
You can measure throughput by using system performance counters, although you may also need to add custom instrumentation code. You also need to use custom code to measure serialization performance. You can capture connection information using the Netstat tool. The performance counters that you use to measure .NET remoting performance are shown in Table 15.7.
Table 15.7: Performance Counters Used to Measure.NET Remoting Performance
Area | Counter |
---|---|
Throughput | .NET CLR Remoting\Remote Calls/sec
ASP.NET Applications\Requests/Sec |
Contention | .NET CLR LocksAndThreads\Contention Rate/sec
.NET CLR LocksAndThreads\Current Queue Length |
Throughput
To measure .NET remoting throughput, use the following counters:
.NET CLR Remoting\Remote Call/sec
Threshold: No specific value.
Significance: Measures the current rate of incoming .NET remoting requests. More than one remote call may be required to complete a single operation. You need to divide the counter with the amount of requests to complete a single operation. This gives you the rate of operations completed per second.
You might need to instrument your code to observe the request execution time.
ASP.NET Applications\ Requests/Sec
Threshold: No specific value.
Significance: If your remote component is hosted in IIS, you can measure the throughput by observing this counter. You need to divide the counter with the amount of requests to complete a single operation. This gives you the rate of operations completed per second.
Serialization Cost and Amount of Data
Serializing data across .NET remoting boundaries can be a major overhead, particularly if you use the SoapFormatter. The amount of data passed over the wire and the processor and memory overhead required to serialize the data can be significant, especially in server applications under heavy load. Large amounts of data serialization can lead to network congestion, processor, and memory bottlenecks.
To optimize and tune serialization, you can measure the costs associated with serializing individual parameters. You can measure byte sizes and the overhead placed on garbage collection. In this way, for a given load you can calculate the total amount of data and the memory overhead by multiplying the values with the number of concurrent users.
using System;
using System.Data;
using System.IO;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
[System.Serializable]
public class MyClass
{
public string name;
public string surName;
public MyClass(){
name="FirstName";
surName="LastName";
}
}
class Class1
{
private static IFormatter iBinForm = new BinaryFormatter();
private static long startMemory, endMemory, gcMemory, actualMemory,
overHeadMemory;
private static double percentOverhead;
private static byte [] byteData = new byte[5000];
[STAThread]
static void Main(string[] args)
{
Console.WriteLine("Press any key to start...");
Console.ReadLine();
Stream serializationStream = new
MemoryStream(byteData,0,byteData.Length,true,true);
serializationStream.Position=0;
MyClass mc = new MyClass();
System.GC.Collect();
System.GC.WaitForPendingFinalizers();
startMemory = System.GC.GetTotalMemory(false);
iBinForm.Serialize(serializationStream,mc);
CalculateOverhead(serializationStream.Position);
DisplayInfo();
serializationStream.Close();
Console.ReadLine();
}
}
Number of TCP Connections
To identify the total number of TCP connections to a given server, you can monitor the TCP\ Connections Established performance counter.
If you need to analyze the connections further and identify which client is consuming the most connections, the state for each connection, and the total number of connections, you can run the following command at the command prompt.
C:\>netstat a n p TCP
The output from this command would be similar to the output shown in Figure 15.5.
Figure 15.5: Output from Netstat.exe
You can observe the TCP connections made from different computers and categorize them based on their IP address.
Interop
This section describes what you need to do to measure interoperability performance and how to capture the key metrics. You can use the measuring techniques discussed in this section to measure P/Invoke and COM interop performance.
What to Measure
To effectively determine interop performance, you need to measure the following:
- Marshaling time per request
- Processor and memory utilization
- Chattiness of marshaled interfaces
How to Measure
You can measure interop performance by using the performance counters shown in Table 15.8.
Table 15.8: Performance Counters Used to Measure Interop Performance
Area | Counter |
---|---|
Processor | Processor\% Processor Time
Processor\% Privileged Time Processor\% Interrupt Time System\Processor Queue Length System\Context Switches/sec |
Memory | Process\Private Bytes
.NET CLR Memory\% Time in GC .NET CLR Memory\# Bytes in all Heaps .NET CLR Memory\# Gen 0 Collections .NET CLR Memory\# Gen 1 Collections .NET CLR Memory\# Gen 2 Collections .NET CLR Memory\# of Pinned Objects .NET CLR Memory\Large Object Heap Size |
Chattiness | .NET CLR Interop\# of marshalling |
Chattiness of Marshaled Interfaces
You can measure interface chattiness by measuring the number of times your code switches from managed to unmanaged code and back again. Use the following counter:
.NET CLR Interop\# of marshalling
Threshold: No specific value.
Significance: This tells you the number of transitions from managed to unmanaged code and back again. If this number is high, determine whether you can redesign this part of the application to reduce the number of transitions needed.
.NET CLR Interop\# of Stubs
Threshold: No specific value.
Significance: Displays the current number of stubs that the CLR has created.
ADO.NET/Data Access
This section describes what you need to do to measure ADO.NET data access performance and how you capture the key metrics.
What to Measure
To effectively determine ADO.NET data access performance, you need to measure the following:
- Connectionpooling. Measure the utilization and effectiveness of pooling.
- Queries. Measure response times and query efficiency.
- Indexes. Measure the effectiveness of index searches.
- Cache. Measure the effectiveness of caching and cache utilization levels.
- Transactions. Measure transactions per second, concurrent transactions.
- Locks. Measure the impact of table-locking and row-locking, average time spent waiting for a lock, and number of deadlocks.
How to Measure
You can measure ADO.NET data access performance by using the performance counters shown in Table 15.9.
Table 15.9: Performance Counters Used to Measure ADO.NET Performance
Area | Counter |
---|---|
Connection Pooling | .NET CLR Data\SqlClient: Current # connection pools
.NET CLR Data\SqlClient: Current # pooled connections .NET CLR Data\SqlClient: Peak # pooled connections .NET CLR Data\SqlClient: Total # failed connects |
Indexes | SQLServer Access Methods\Index Searches/sec
SQLServer Access Methods\Full Scans/sec |
Cache | SQL Server: Cache Manager\Cache Hit Ratio
SQL Server: Cache Manager\Cache Use Counts/sec SQL Server: Memory Manager\ SQL Cache Memory(KB) Memory\ Cache Faults/sec |
Transactions | SQL Server: Databases\Transactions/sec
SQL Server: Databases\Active Transactions |
Locks | SQL Server: Locks\ Lock Requests/sec
SQL Server: Locks\ Lock Timeouts/sec SQL Server: Locks\Lock Waits/sec SQL Server: Locks\ Number of Deadlocks/sec SQL Server: Locks\Average Wait Time (ms) SQL Server: Latches\Average Latch Wait Time(ms) |
Connection Pooling – SqlConnection
To monitor the connection pool of .NET Framework data provider for SQL Server, you can monitor the following performance counters:
.NET CLR Data\ SqlClient: Current # connection pools
Threshold: No specific value.
Significance: Current number of pools associated with the process.
.NET CLR Data\ SqlClient: Current # pooled connections
Threshold: No specific value.
Significance: Current number of connections in all pools associated with the process.
.NET CLR Data\SqlClient: Peak # pooled connections
Threshold: No specific value.
Significance: The highest number of connections in all pools since the process started.
.NET CLR Data\SqlClient: Total # failed connects
Threshold: No specific value.
Significance: The total number of connection open attempts that have failed for any reason.
More Information
For more information about a counter reset problem, see Knowledge Base article 314429, "BUG: Performance Counters for SQL Server .NET Data Provider Are Not Reset," at https://support.microsoft.com/default.aspx?scid=kb;en-us;314429.
Connection Pooling – OleDbConnection
Currently there is no direct way to measure the effectiveness of OLEDB and ODBC pools by using System Monitor because the counters provided are not reliable. However, you can monitor the number of logins per second with the SQL Server: General Statistics\Logins/sec counter. You can also monitor user connections on the database server to evaluate pool performance.
You should observe that SQL Server: General Statistics\Logins/sec drops to zero. This indicates that connections are getting repeatedly reused and that the pool is working effectively.
Indexes
To measure index performance, use the following counters:
SQL Server: Access Methods\Index Searches/sec
Threshold: No specific value.
Significance: Number of index searches. Index searches are used to start range scans, single index record fetches, and to reposition within an index.
SQL Server: Access Methods\Full Scans/sec
Threshold: No specific value.
Significance: The rate of full table or full index scans. Lower numbers are better.
To tune indexes, use the Index Tuning Wizard that comes with SQL Server 2000.
Cache
To measure caching, use the following counters:
SQL Server: Cache Manager\Cache Hit Ratio
Threshold: No specific value.
Significance: Ratio between cache hits and lookups.
SQL Server: Cache Manager\Cache Use Counts/sec
Threshold: No specific value.
Significance: Times each type of cache object has been used.
SQL Server: Memory Manager\SQL Cache Memory (KB)
Threshold: No specific value.
Significance: Total amount of dynamic memory the server is using for the dynamic SQL cache.
Memory\Cache Faults/sec
Threshold: This indicates how often the operating system looks for data in the file system cache but fails to find it.
Significance: This value should be as small as possible. A high rate of cache faults may indicate insufficient memory or poorly organized or heavily fragmented disks.
Transactions
To measure transactions, use the following counters:
SQL Server: Databases\Transactions/sec
Threshold: No specific value.
Significance: Number of transactions started for the database. This is the primary indicator of database throughput.
SQL Server: Databases\Active Transactions
Threshold: No specific value.
Significance: Number of active transactions for the database.
Locks
To measure the impact of locks in the database, use the following counters:
SQL Server: Locks\ Lock Requests/sec
Threshold: No specific value.
Significance: Number of new locks and lock conversions requested from the lock manager.
SQL Server: Locks\ Lock Timeouts/sec
Threshold: No specific value.
Significance: Number of lock requests that timed out. This includes internal requests for NOWAIT locks.
SQL Server: Locks\Lock Waits/sec
Threshold: No specific value.
Significance: Number of lock requests that could not be satisfied immediately and required the caller to wait before being granted the lock.
SQL Server: Locks\ Number of Deadlocks/sec
Threshold: No specific value.
Significance: Number of lock requests that resulted in a deadlock. A typical reason for this could be interference between long-running queries and multiple row updates. This number has to be very low. This translates to significant extra work because a deadlock implies that there must be a retry or compensating action at some higher level of the business logic.
High values indicate that there is a scope to improve your design that manages the transaction isolation levels and queries.
SQL Server: Locks\Average Wait Time (ms)
Threshold: No specific value.
Significance: The average amount of wait time (milliseconds) for each lock request that resulted in a wait.
SQL Server: Latches\Average Latch Wait Time (ms)
Threshold: No specific value.
Significance: The average amount of time that a request for a latch had to wait within the database. Latches are lightweight, short-term row locks, so higher numbers indicate contention for resources.
Summary
Having set performance objects early in your application's design phase, you begin to measure by collecting metrics. You continue to measure throughout the application life cycle to determine whether your application's performance is trending toward or away from its performance goals.
This chapter has shown you what the key metrics are that you need to capture for the CLR and managed code in general and also for specific .NET technologies such as ASP.NET, Web services, Enterprise Services, COM interop, and ADO.NET data access. In each case, the chapter has shown you what to measure and how to measure the key metrics.
Additional Resources
For more information about measuring performance, see the following resources in this guide:
- Chapter 2, "Performance Modeling"
- Chapter 16, "Testing .NET Application Performance"
- Chapter 17, "Tuning .NET Application Performance"
See the following How Tos in the "How To" section of this guide:
- "How To: Use CLR Profiler"
- "How To: Monitor the ASP.NET Thread Pool Using Custom Counters"
- "How To: Time Managed Code Using QueryPerformanceCounter and QueryPerformanceFrequency"
- "How To: Use Custom Performance Counters from ASP.NET"
- "How To: Use EIF"
- "How To: Use ACT to Test Performance and Scalability"
- "How To: Perform Capacity Planning for .NET Framework Applications"
For further reading, see the following resources:
- Performance Testing Microsoft .NET Web Applications, by Microsoft ACE Team
- Performance Engineering of Software Systems, by Connie U. Smith
Retired Content |
---|
This content is outdated and is no longer being maintained. It is provided as a courtesy for individuals who are still using these technologies. This page may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist. |