Bugslayer

Measuring the Impact of View State

John Robbins

Code download available at:Bugslayer2007_11.exe(209 KB)

Contents

View State
Debugging Hints
Goals
Tracing View State
Implementation Highlights
Further Ideas

Isn't it funny how the Microsoft® .NET Framework is thought of as an environment where you don't have to think about memory? Of course, what's one of the number one issues that continues to plague managed applications today? Memory! Why? When the garbage collector runs to reclaim memory, the common language runtime (CLR) suspends all the threads in the application—and no work gets done. When you're not getting any work done, you have a performance problem.

No matter how much you care about your .NET Framework memory usage, a couple of major problems interfere with actually getting a useful picture of that memory. The first is that though the .NET Framework does a tremendous amount of work on your behalf, the documentation is not necessarily helpful in figuring out what work is being performed and how much memory is being used. Fortunately, Lutz Roeder's Reflector for .NET (aisto.com/roeder/dotnet) helps considerably.

The second problem with observing memory usage is that the main development tool for .NET, Visual Studio®, does not show you anything about memory. That's not the fault of the Visual Studio team, but a failing in the original CLR architecture, which doesn't expose a means through the CLR Debugging API to see what generation a particular object is in. While it's possible to use the CLR Profiling API to keep track of object generations, it takes tremendous effort and does not work when you're debugging. As I wrote in the June 2003 (msdn.microsoft.com/msdnmag/issues/03/06/Bugslayer) and again in the March 2005 (msdn.microsoft.com/msdnmag/issues/05/03/Bugslayer) Bugslayer columns, you can use the Son of Strike (SOS) WinDBG extension to see all the memory information you want, but it's probably the most painful tool to use since the original Windows® 95 kernel debugger.

In this episode of Bugslayer, I want to present a tool that lets you look at one of the most insidious performance killers in an ASP.NET app: view state. As you'll see, being able to see your view state usage easily on a production server can mean the difference between thirty minutes of debugging and six weeks of guessing.

View State

In an ASP.NET page, view state is an extremely nifty mechanism whereby the state of the controls on the page is stored in a hidden value within the HTML code sent to the client. This avoids maintaining the state of the page in the ASP.NET worker process, which would put even more memory pressure on the server. If you ever worked on an ISAPI filter written in native C++ back in the Jurassic period of the Internet (1996 or so), you remember how much effort went into keeping track of page, session, and application state. If you don't have much ASP.NET experience, you'll want read Dino Esposito's February 2003 Cutting Edge column on view state as a primer (msdn.microsoft.com/msdnmag/issues/03/02/cuttingedge).

The problem with view state is that you really have no idea how big that view state value is at any given time. This is especially true when there's a GridView control on the page. The ease of use in getting that database information on the page comes at a cost. The good news is that in ASP.NET 2.0, the Microsoft developers worked hard to bring the size of the view state down compared to ASP.NET 1.1. However, in the last couple of months I've helped debug several large ASP.NET 2.0 applications where the performance issues were caused by jumbo-size view states on certain pages.

Debugging Hints

Before we get into the tool and code, I want to discuss a few of my debugging tricks. When developers call me about an ASP.NET application that is having a performance problem, they invariably say, "The application was working fine, but now it is really slow."

My first question is, "Are you using any GridViews in your application?" As most ASP.NET applications are front ends to databases, that's pretty much a given, but not always. If they are not using GridViews, I need to look at a different path.

The second question I ask is, "Can I look at your database stored procedure history in version control?" Sadly, at this point, far too many development teams gulp hard and tell me they don't have their stored procedures in version control. The last time I looked, a SQL stored procedure was code, and if you follow the common-sense rules of development, all code goes into version control. In fact, Microsoft makes it exceedingly easy with the Visual Studio 2005 Team Edition for Database Professionals. If you don't have your stored procedures in version control, you need to stop reading right now and add them.

The reason I ask to look at their stored procedures in version control is that when the application is running fine and suddenly starts having issues, I need to find the changes between the previous and current editions. I can't begin to tell you how many times I've looked through the version control change logs for the stored procedures and seen where someone has changed a query to be something along the lines of:

select * from table

When you're pumping that output to a GridView control (which relies heavily on the view state) and the database contains six million records, you have a serious view state problem!

Though view state is easy to use, it's nearly impossible to see when you've had a problem. Most developers know you can see the view state for a page by turning on ASP.NET tracing. Unfortunately, ASP.NET tracing only stores a limited number of traces and it can be extremely slow on a production server. Your alternative was to have each user right-click in the browser, select View Source from the context menu, and start manually counting characters. Clearly, something better was needed: a simple way to keep track of the largest view state for all pages on a production server so you could have your data in one place to make intelligent debugging decisions.

Goals

When I first started thinking about the challenges of a tool that lets you easily see your view state usage, the prime goal was that whatever I came up with had to be usable in a production environment. That meant the code had to be both fast and frugal with memory. Therefore, I made the decision to store only the largest view state size for each particular page. If I kept all of the view state sizes for every page, I could easily be forced to store millions of records. By storing only the largest view state for a given page, I drastically cut down the storage requirement, ensuring that there are never more data entries than there are number of .aspx files in the ASP.NET application.

I also wanted to make looking at the view state data simple and understandable, even for non-developers. While seeing the total view state size for a page is great, an additional requirement I added was an option to store and display the view state sizes for all of the controls on a page. That would make it much easier to identify the control that contributed most to a problem.

I wanted the option to see the view state size on HTTP GET requests as well. While nearly all view state problems are related to HTTP POST requests, users can store the full URL to a particular page and go directly to that page. That's a rare occurrence, but I wanted to include this option for completeness.

Finally, I wanted to ensure easy deployment for the tool. As a sub-goal, it would be ideal if my solution required no source code changes.

The source code download for this month's column includes the assembly that contains all the magic: Bugslayer.Web. Also included are the complete unit tests so you can play around with the assembly and see how it works. For production servers, you'll want to install Bugslayer.Web.DLL into the Global Assembly Cache (GAC), as that will let you load the code from anywhere on the machine.

There are two parts to Bugslayer.Web: an HttpModule and an HttpHandler. If you've read much about the ASP.NET processing pipeline, you can probably guess that the HttpModule is where the data collection occurs and the HttpHandler is what displays the data. And this means there are no code changes at all on your part to use Bugslayer.Web.

To configure your ASP.NET application to use the HttpModule, you'll need to add the <httpModules> element like the following to your web.config file:

<?xml version="1.0"?> <configuration> <system.web> <httpModules> <add name="ViewStateStats" type="Bugslayer.Web.ViewStateStatisticsModule"/> </httpModules> </system.web> </configuration>

Setting up the HttpHandler is nearly as easy. In your web.config file, add the <httpHandlers> element as follows:

<?xml version="1.0"?> <configuration> <system.web> <httpHandlers> <add verb="*" path="viewstatestats.axd" type="Bugslayer.Web.ViewStateStatisticsHandler" validate="false" /> </httpHandlers> </system.web> </configuration>

Notice that the Bugslayer.Web.ViewStateStatisticsHandler is mapped to viewstatestats.axd. I chose to use the .axd extension because IIS already understands it as being an ASP.NET extension type. By setting the path attribute to a specific file name, ASP.NET will call that specific HttpHandler, but requests for other file names are routed to the ASP.NET internal .axd handler.

Tracing View State

After installing Bugslayer.Web and adding the entries to web.config, you'll be recording the largest view state byte count per page. While collecting the data is nice, you do need to be able to view it and, as discussed, that's what ViewStateStats.axd is for. Figure 1 shows example output. The format might look familiar as I borrowed the style from the ASP.NET tracing output. As you can see, the default is from largest to smallest view state. To sort on the URL or time, which both sort ascending, click on the column headers.

Figure 1** Example ViewStateStats.axd Output **(Click the image for a larger view)

You can control the view state data collection in the upper portion of ViewStateStats.axd page. The most interesting option is the second link, Collect Control View State Sizes. If the state is disabled (the default option) you'll only collect total sizes for the page. Clicking on the link will toggle the collection state to keep the list of child controls and their individual view states for the largest total view state for each page. While that will cause a performance hit and take more memory, there are plenty of times when you do need to figure out which control is hogging the view state. Figure 2 shows an example ViewStateStats.axd page with the controls expanded.

Figure 2** ViewStateStats.axd Control Tree Output **(Click the image for a larger view)

Just like the ASP.NET tracing output, the control tree is displayed along with the name of the control and its view state size. If you're wondering where the render size bytes and ControlState size bytes values are, there's no way for me to get the render data, and I felt the ControlState size calculations would be too slow. I'll discuss why in the implementation section. However, since the view state is the major performance killer, the truly important control contribution is there. Note that the individual control sizes will not add up to the total view state size shown next to the URL and time. Since the view state is Base64 encoded, the encoding adds additional bytes to the overall view state.

The first link under the ViewStateStats.axd title, Clear all view state statistics, flushes all the existing saved values and starts fresh. The final link, Collect View State Sizes for HTTP GET Requests, turns on reporting for HTTP GET requests. As nearly every worry you have about view state is an HTTP POST, it's rare you'll need to turn on this option. However, if you do need to see view state sizes for both HTTP GETs and HTTP POSTs, you'll have the ability. If you toggle this option on, after each URL in the ViewStateStats.axd page you'll see a (GET) or (POST) as appropriate. When I discuss how I calculate the HTTP GET view state size, you'll see that there are performance implications for recording this data.

Implementation Highlights

When I first started toying around with the idea of recording the view state size for a whole application, I prototyped a derived System.Web.UI.Page class. It's trivial to get the view state string with the following C# code:

String viewState = Request.Params [ "__VIEWSTATE" ];

The problem with using a derived Page is that, for many existing ASP.NET applications, it would require you to change your code to see the view state across your application. Since HttpModules allow your code to participate in the normal ASP.NET pipeline, that was the perfect solution. Because I only wanted to inspect the page after it was ready to be sent down to the user, I set an EndRequest event handler in my IHttpModule.Init implementation method.

One interesting discovery that I made while reading about how the ASP.NET view state worked was following web.config setting:

<pages maxPageStateFieldLength="XX"/>

Here, XX is the number of maximum bytes for breaking up the __VIEWSTATE value into chunks. You can also set this with the System.Web.Ui.Page.MaxPageStateFieldLength property.

ASP.NET 2.0 introduced this setting in order to help break up huge __VIEWSTATE values because some firewalls and proxies choke on gigantic strings on the page. Given that gigantic strings are something you want to avoid, and the whole point of this column, you might think that setting the maxPageStateFieldLength could help reduce some size. Not only does it not help, it actually bloats your pages going down to the browser.

Say you have a view state like the following on the page:

<div> <input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="/wEPDwULLTE1ODEyMjM1MTgPZBYCAgMPZBYCAgcPEGQPFgNmAg­ECAhYDEAUBYQUBYWcQBQFiBQFiZxAFAW MFAWNnZGRkQIU7yEIZAT5uuif4cZcFHBJgZ48=" /> </div>

Setting maxPageStateFieldLength to 50 yields the following, which has far more actual text on the page:

<div> <input type="hidden" name="__VIEWSTATEFIELDCOUNT" id="__VIEWSTATEFIELDCOUNT" value="3" /> <input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="/wEPDwULLTE1ODEyMjM1MTgPZBYCAgMPZBYCAgcPEGQPFgNmAg" /> <input type="hidden" name="__VIEWSTATE1" id="__VIEWSTATE1" value="ECAhYDEAUBYQUBYWcQBQFiBQFiZxAFAWMFAWNnZGRkQIU7yEIZ" /> <input type="hidden" name="__VIEWSTATE2" id="__VIEWSTATE2" value="AT5uuif4cZcFHBJgZ48=" /> </div>

While there are probably a few readers who have set maxPageStateFieldLength, I doubt many of you are using it. If you are breaking the view state into chunks to get around firewall and proxy issues, I strongly suggest you consider redesigning the page or storing your view state on the server to avoid sending huge amounts of data over the wire.

My HttpModule, which is implemented in ViewStateStatisticsModule.cs, properly handles the case where maxPageStateFieldLength has been set. Since I have to enumerate each of the __VIEWSTATE* parameters, the code I use is a little slower than accessing a single __VIEWSTATE parameter.

Because the data collection and display are logically coupled, I decided to group them behind the IViewStateProcessor interface, and then decouple all of that functionality from my HttpModule and HttpHandler. The IViewStateProcessor interface provides methods to process both HTTP GET and HTTP POST requests, and to display the data. My idea was to make it easy to provide additional data collection and display without having to touch most of the code in Bugslayer.Web.dll.

The class derived from IViewStateProcessor must be in memory at all times, so I store it in a static field inside my ViewStateStatisticsModule class. The ViewStateStatisticsHandler class simply accesses a static property in ViewStateStatisticsModule. Right now the code supports only a single IViewStateProcessor derived class. A nice addition would be to add support for simultaneously using additional classes.

To specify your own custom IViewStateProcessor-derived class instead, you'll need to override the HttpApplication.Init method. In there you'll access the ViewStateStatisticsModule through HttpApplication.Modules["ViewStateStats"]. Create an instance of your custom IViewStateProcessor-derived class and set the ViewStateStatisticsModule.ViewStateProcessor to your instance.

The default IViewStateProcessor-derived class, BiggestPageProcessor, is where all the real work occurs for collecting and displaying the data you saw inFigures 1 and 2. The collection data is stored in a Dictionary class with the URL itself as the key for fast lookup. The interesting part of the collection code was figuring out how to get the view states for individual controls.

As everyone is used to seeing the view state sizes for each individual control, I wanted to ensure my numbers matched for correctness. That meant I needed to figure out where in the ASP.NET tracing code the action happened. The first stop on the Reflector party boat was System.Web.Handlers.TraceHandler, which is the HttpHandler registered for trace.axd. About 15 minutes of exploring the decompiled code and looking for the referenced data types got me pointed at System.Web.Ui.Page.ProcessRequestMain, which performs the actual tracing if Page.Context.TracingEnabled is true. After bouncing through Page.BuildPageProfileTree, I finally got to Control.BuildProfileTree, which is where all the action happens to get the controls and their data into the trace output. Figure 3 shows the decompiled code.

Figure 3 Decompiled Control.BuildProfileTree Method

protected void BuildProfileTree(string parentId, bool calcViewState) { int viewStateSize; calcViewState = calcViewState && !this.flags[4]; if (calcViewState) { viewStateSize = this.EstimateStateSize(this.SaveViewState()); } else { viewStateSize = 0; } int controlStateSize = 0; if (((this.Page != null) && (this.Page._registeredControlsRequiringControlState != null)) && this.Page._registeredControlsRequiringControlState.Contains( this)) { controlStateSize = this.EstimateStateSize(this.SaveControlStateInternal()); } this.Page.Trace.AddNewControl(this.UniqueID, parentId, base.GetType().FullName, viewStateSize, controlStateSize); if ((this._occasionalFields != null) && (this._occasionalFields.Controls != null)) { int count = this._occasionalFields.Controls.Count; for (int i = 0; i < count; i++) { this._occasionalFields.Controls[i].BuildProfileTree( this.UniqueID, calcViewState); } } }

The interesting part of the code for view state is the first 10 lines of the method. The first bit of code that really threw me is the !this.flags[4] reference because Reflector shows the flags field as private. Using Reflector's lifesaving Analyzer, I jumped to the flags field and clicked on the "Used By" tree element. Scanning for anything related to view state showed that the Control.EnableViewState property getter was in the list. Hopping over to the decompiled EnableViewState getter method showed that the method actually returns !this.flags[4], so that mystery was solved.

The EstimateStateSize just creates an ObjectStateFormatter, calls the Serialize method, and returns the size of the serialization bytes, so there was no trouble duplicating that. Things are definitely more interesting with the call to the protected this.SaveViewState method and I wanted to call it from my HttpModule. Given that .NET reflection is the secret sauce to reach out and call anything, I used a dash of it to ensure that I was going to get the same numbers as the ASP.NET tracing system. Figure 4 shows the core code I wrote for calculating individual control view state sizes.

Figure 4 Core Control View State Size Calculation Code

private PageViewState CreatePageViewState ( String url , DateTime timeStamp , Int32 viewStateLength , String httpRequestType ) { PageViewState returnValue = new PageViewState ( url , timeStamp , viewStateLength , httpRequestType ); if ( true == collectControlData ) { Page currentPage = HttpContext.Current.Handler as Page; if ( null != currentPage ) { CalculateControlSizes ( currentPage , 0 , returnValue.ControlList ); } } return ( returnValue ); } private static void CalculateControlSizes ( Control ctl , Int32 indentLevel , List<ControlViewState> ctlList ) { Int32 viewStateSize = EstimateSize ( ctl ); ControlViewState state = new ControlViewState ( indentLevel , ctl.UniqueID , ctl.GetType ( ).FullName , viewStateSize ); // Add it to the list. ctlList.Add ( state ); for ( int i = 0 ; i < ctl.Controls.Count ; i++ ) { CalculateControlSizes ( ctl.Controls [ i ] , indentLevel + 1 , ctlList ); } } private static Int32 EstimateSize ( Control ctl ) { // In Control.BuildProfileTree, the check is done against flag[4], // which is the same as the implementation of EnableViewState. if ( false == ctl.EnableViewState ) { return ( 0 ); } Object saved = CallSaveViewStateMethod ( ctl ); if ( null == saved ) { return ( 0 ); } ObjectStateFormatter osf = new ObjectStateFormatter ( ); Int32 ret = osf.Serialize ( saved ).Length; return ( ret ); } private static Object CallSaveViewStateMethod ( Control ctl ) { // This is way ugly, but the SaveViewState method is protected on // the Control class so in order to get the total size, I need to // call it through Reflection. Object ret = null; BindingFlags flags = BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic; Type type = ctl.GetType ( ); MethodInfo m = type.GetMethod ( "SaveViewState" , flags ); if ( null != m ) { ret = m.Invoke ( ctl , null ); } return ( ret ); } private Int32 SumControlSizes ( Control ctl ) { Int32 viewStateSize = EstimateSize ( ctl ); for ( int i = 0 ; i < ctl.Controls.Count ; i++ ) { viewStateSize += SumControlSizes ( ctl.Controls [ i ] ); } return ( viewStateSize ); }

Once I figured out the trick to calculating the view state just like the ASP.NET tracing code, I then needed to start testing to see whether the view state values were the same when I was asking for them inside the HttpModule.EndRequest event handler. The good news is that, after running all sorts of pages through Bugslayer.Web, I never saw a discrepancy between what tracing reported and what I calculated.

If you're curious why I didn't include render size bytes in the display, it's because there's no way for me to calculate those values. That happens during page rendering and my HttpModule is far too late in the pipeline. As for the ControlState size bytes, I chose not to calculate them for performance reasons. Figure 3 shows the work necessary to do the math. However, I would have had to use lots of slow reflection to make those calls for very little payback.

Eagle-eyed readers might have noticed that I discussed how getting the view state through Request.Params["__VIEWSTATE"] is accessing a form variable that does not exist with an HTTP GET request. However, if you look at the source code for a page in the browser, the __VIEWSTATE variable is certainly there, so I wanted to find a way to get the data in case a user goes to a page directly in the browser address bar.

After all sorts of poking around, the best solution I could come up with was to do the calculations myself by looping through the controls and getting each individual view state size. While not ideal, it does get close to the actual view state size without getting into full BASE64 encoding and the additional overhead. Because of this limitation on HTTP GET processing, I left the option for monitoring it off by default.

Further Ideas

Overall, I'm quite happy how the view state monitoring tool turned out. By making it very easy to integrate into an existing production application, my goal of looking for those fat view states should become a good bit easier to realize. When you've got a silent performance killer in the view state, being able to look at the page view state sizes easily means you'll be debugging faster so you can work on new features and the fun parts of why we all got in this business.

As with all Bugslayer columns, there are always additional features you can add to Bugslayer.Web to track even more information about your view state situations.

The version of the tool available with this column only tracks view state that's on the page in the form of a __VIEWSTATE hidden field because that's the worst performance killer of all. You may want to consider extending Bugslayer.Web to track view state information stored on the server. That will require more work as you'll need to provide a custom Page-derived class to override the Page.SaveViewState method.

In the sample, the view state storage is only on a per HttpApplication basis. A nice addition would be to track the largest view state across a Web farm.

As I wanted to make the control tree like what ASP.NET tracing shows, I'm including controls that have no view state data. A quick optimization would be to collect only those controls that contribute to view state. Better yet, make it a toggle-able option in the ViewStateStats.axd handler.

If you're a real HTML and JavaScript wizard, you could certainly make the ViewStateStats.axd page display better. I chose the simplest form of display, but having expandable and collapsible control display would make it easier to see very large Web site data.

Finally, on a personal note, I would dearly like to thank all the readers for reading 10 years of Bugslayer columns. All your questions and comments over the years have made writing these columns far more fun than you can ever imagine!

Tip 80 Recently, I was working on a problem where an application pool in IIS was recycling unexpectedly. In my desperate Internet searching I ran across a great blog entry from Johan Straarup, "Common reasons why your application pool may unexpectedly recycle" at blogs.msdn.com/johan/archive/2007/05/16/common-reasons-why-your-application-pool-may-unexpectedly-recycle.aspx. It's a great summary of all the reasons you're wondering about when W3WP.EXE has gone for a walk in the park. Johan's blog is one you need to subscribe to.

Tip 81 Want to become a better debugger? Roberto Farah, who has forgotten far more about WinDBG than any of us could hope to know, put together one of the best lists of debugging books I have ever seen at blogs.msdn.com/debuggingtoolbox/archive/2007/06/08/recommended-books-how-to-acquire-or-improve-debugging-skills.aspx. No matter what type of development you're doing on Windows, Roberto has you covered. In his blog, Roberto is doing things with WinDBG scripting that will completely blow your mind! My personal favorite is his WinDBG script to cheat at Mine Sweeper (blogs.msdn.com/debuggingtoolbox/archive/2007/03/28/windbg-script-playing-with-minesweeper.aspx). Now I win every time!

Tip 82 If you've read past Bugslayer columns, you know I'm a freak about MSBuild tasks (msdn.microsoft.com/msdnmag/issues/06/03/Bugslayer). You should automate the world through MSBuild. The SDC Tasks have been updated to include over 300 tasks, including creating Web sites, application pools, and Active Directory users, running FxCop, configuring virtual servers, creating zip files, configuring COM+, creating folder shares, installing into the GAC, configuring SQL Server, and configuring BizTalk® Server 2004 and BizTalk Server 2006. To control life as you know it from an automated build, download the new tasks from codeplex.com/sdctasks.

Send your questions and comments for John to slayer@microsoft.com.

John Robbins is a cofounder of Wintellect, a software consulting, education, and development firm that specializes in the .NET and Windows platforms. His latest book is Debugging Microsoft .NET 2.0 Applications (Microsoft Press, 2006). You can contact John at wintellect.com.