Troubleshooting and Repair: Power Down, Speed Up
Ensuring your application load is running efficiently can save you time and money with respect to infrastructure operations.
Wes Miller
There are so many questions about the world that remain unanswered. How does the brain really work? Who built the pyramids—and how? What’s the deal with the platypus? Why is my computer running so slow? The brain, the pyramids and the platypus may remain mysteries, but let’s dig in to why the computer is so slow.
Computers with inefficient software cost money. These costs come from reduced user productivity, increased electricity consumption, reduced battery life, thermal wear and tear, disk failure and more. Ignoring problematic software never works—it often leads to much larger problems in terms of cost and time.
When Microsoft shipped Windows XP, it spent a fair amount of time ensuring that the Windows notification area (what many people call “the tray”) had a manageable UI. Unfortunately, so many software developers misused it that it wasn’t atypical to see a new computer with more than 10 icons in the notification area running at logon.
Everything has a cost—and each one of those apps in the notification area has a cost beyond just desktop real estate. Each one represents at least one running application. And every version of Windows adds even more services than the previous version. In short, there’s a lot of stuff running on your computer, whether it comes from an OEM, a standard corporate desktop image or whether you’ve installed Windows yourself.
Faster, Faster
Besides “Why won’t Windows start?”, one of the more common questions I get asked is, “What is making this computer so slow, and what can I do to solve it?”
If you have two computers that began with the same image (especially when only one is exhibiting symptoms), it’s always an interesting starting point to compare the two systems to determine the variations between them. Running the tools we’ll discuss below can often help you spot the problem process much easier.
If you think about resources on your computer, simplistically there are four types of resources that can affect your overall ability to get work done:
1. Memory (RAM)
2. CPU
3. Disk I/O
4. Network I/O
To help you perform this process, I’m going to be using the Sysinternals Suite quite a bit. If you have it, let’s get started. If you don’t—get it from the Windows Sysinternals download page.
Before we begin, let’s get a foundation of what’s running on your computer. With the Sysinternals tools, there are two options. Both depend on how you want to view the information, and where you are in relation to the computer you need to diagnose.
Process by Process
The first option is Process Explorer (Procexp.exe), which is far and away the more powerful of the two options, but it only runs locally (versus the next tool, as we’ll see in a moment). Process Explorer can be best thought of as a powerful replacement for Windows Task Manager. It lets you see at a glance all of the processes running on your system in considerable detail—whether they’re yours or that of another logged-on user (if you’re using Fast User Switching or Terminal Services/Remote Desktop).
Before we begin, ensure that you’re running Procexp with all processes visible by re-launching it using File | Show Details for all Processes (which will prompt you if you’re running Windows with UAC enabled).
By sorting the Process column, you can specify whether the processes should be listed in ascending or descending order, or in a process tree based upon what application launched each one. Bear in mind that processes can start, launch another application and exit—leaving a false impression as to what process really launched the child.
The easiest way to see what process is taking the most CPU time is by hovering over the CPU graph. It will show you what process it is. You can then use Ctrl+F to find the process.
Another question I’m asked all the time is, “What is System Idle Process, and why is it using all the CPU on my computer?” It’s not using it. That process can best be thought of as a placeholder for unused CPU time. It’s not even really an actual process. If you haven’t, you may want to read “Windows Internals” (Microsoft Press, 2009) for a strong foundation on Windows internal fundamentals.
There are two options available:
- View | Update Speed lets you change how often Procexp changes the UI to reflect updates (which are relatively instantaneous).
- If you watch for the high CPU usage process to spike to the top, then press your spacebar. Procexp will stop updating the UI entirely, letting you see the suspect process.
You can expose high memory-consuming processes pretty easily in this manner as well. If you don’t recognize the process by name (as you may not if it’s malware), selecting the process and clicking Process | Window will identify the application if it has any exposed UI. Services have no UI, and applications can elect not to (as some legacy applications that really should be services tend to do).
To Service and Protect
This diagnostic process works well for applications running in user context, but what about services or drivers? Services can be more problematic to isolate, because there’s a growing tendency to run them in a “shared process” to save resources and increase security.
Windows offers a built-in method to perform this, using a service called svchost.exe (you’ll note quite a few of these on any Windows system beginning with Windows XP). Many of these are built-in Windows services, but third parties can use this infrastructure as well. In fact, there’s a growing tendency for malware to use svchost as a host.
You can still identify instances of svchost that are causing high CPU or RAM utilization. Double-click on each instance, select the Services tab, and you can view the DLLs that are running in that shared process, and see which one is actually consuming the resources.
Finally—what about drivers? Don’t ignore drivers as a potential source of system performance degradation. They complete the lowest-level tasks in Windows. All too often, they can cause problems.
Drivers are often tasked with watching other processes like antimalware, or watching for file or directory notifications like antimalware or replication/synchronization. It’s all too easy to write a misbehaving driver or a driver that misbehaves only in the presence of another driver. That’s worse because it’s harder to diagnose.
So where do drivers live? Like services hosted in svchost.exe, drivers also run in a shared process. If you double-click on the System process and click on the Threads tab, you can see the thread stack it has loaded. You will see many drivers (*.sys files) loaded here.
As with the individual process view, you can sort them by CPU time. Doing so will expose threads that a particular driver is executing, which are taking considerable CPU time. You can then search your system for the driver, search the Internet, or contact the vendor to find out if they’re aware of the issue or if there’s a newer version of the driver software.
If you’re connecting remotely to the computer you want to diagnose, you can connect via the network, but you can’t connect with TS/Remote Desktop. Then it’s PsTools to the rescue. You’ll find PsList helpful here. PsList, as the name would imply, is a command-line PsTool (included in the Sysinternals Suite) that lists all of the processes running on a local or remote machine.
Like all the PsTools, you can run it remotely via File and Printer Sharing, as long as it’s enabled and you have credentials as a member of the local Administrators group. As with Procexp, you can see the list of all processes running at the time you executed PsList. You’ll find the information PsTools provides is more limited, but in cases like this where you’re diagnosing a problem process, there’s generally enough information to complete the task.
To execute PsList against a local machine, simply run PsList.exe. The –t switch lets you view in a tree-view similar to Procexp. Adding –s (an interval in seconds) will autopoll at the interval you define. This is the opposite of Procexp, where you can opt to slow down refreshing from the default. All the PsTools have similar remote usage as shown here:
PsTool.exe \\systemname -u username –p password
You can use Procexp to kill processes on a local machine (but be careful—killing random processes can cause system corruption or software failure), but not PsList. You can also use PsKill to kill a process by Process ID (PID) or by name (it will kill all processes with the same name), even remotely.
Diagnosing disk I/O problems (especially applications making your system constantly seek data on the hard disk) is complex and in some cases may be difficult to fully diagnose. However, running Process Monitor (Procmon.exe) can often rapidly expose services or applications reading or writing to the hard disk too frequently. While many applications may have a legitimate reason for consuming disk I/O, such as local search utilities, the reality is that much disk I/O is wasted on repetitive tasks that could possibly be handled in better ways.
When you’ve identified the process that looks as if it’s reading or writing too often, you can stop running Procmon, switch back over to Procexp and use Ctrl+F to find the same process. Be sure if you’re searching by name to verify that it’s the same PID. Some processes, such as explorer.exe and svchost.exe, will have multiple instances. Once you’ve identified the process-consuming disk I/O, you can contact the vendor and see if there’s any update.
When it comes to network communication, TCPView is another Sysinternals tool to determine processes connecting to the network (inbound or outbound), including port information. You can also use Procmon, as it will show you network connectivity as well. Most importantly, it will expose high rates of network communication in real time as they occur. Similar to disk I/O, hopping over to Procexp will let you perform deeper diagnosis, while still seeing network activity (TCP/IP tab for each process).
Standing Alone
These methods rely on you having some connectivity to the system, but what if you don’t? What if it’s your sibling’s system, and she lives thousands of miles away? It’s actually easier than you might think. First, have the remote person download the Sysinternals tools. They’re all easy to use, especially the way we’ll be using them.
Once she has the tools downloaded, have her run AutoRuns. This lets her provide you with a snapshot of all executables running on her system, in the form of an *.arn file—a file type Mark Russinovich developed explicitly for this task, to enable remote analysis of autostarts.
With her logged on as an administrator, have her generate the ARN file for you by clicking File | Save. Why did we capture the ARN file? That’s the best way to determine what’s starting the process.
Whether connecting locally or remotely, you’re finding out exactly what’s launching each application, and what services or other “AutoStarts” are running on the computer that you might not know about. AutoRuns has quite a few categories of items that are autostarted. The most interesting for this exercise are probably the logon, service and drivers items.
Once AutoRuns has completely populated the UI, select Entry | Hide Microsoft and Windows Entries. Click F5 and you’ll see the non-Microsoft entries. This is often a good way to start diagnosing suspect services or applications from a remote system.
Next, have her run Procexp for you. Run this as an administrator as well, and select File | Save As to save the detailed tree view as a text file.
Then have her run Procmon for a period of time, while her system is “symptomatic.” Realistically, you’re looking at less than five minutes here for diagnosis.
Finally, have her send you the ARN, process tree and Procmon log files. The Procmon file will undoubtedly be too large to send via e-mail. Use a shared file transfer tool such as Dropbox.
Once you have the files, you have most of the information we used to diagnose your local performance degradation. Taking the Procmon’s logs (which will provide the most information, and filter it down in the easiest manner possible) and Procexp’s process tree, you can often find the culprit just as easily in a remote scenario as locally.
Diagnosing problematic applications under Windows isn’t impossible. With the Sysinternals tools, the Debugging Tools for Windows and some practice, you’ll soon be diagnosing problems deep within Windows. You’ll save time, energy and perhaps even your sanity.
.jpg "Wes Miller")
Wes Milleris the director of Product Management at CoreTrace (CoreTrace.com) in Austin, Texas. Previously, he worked at Winternals Software and as a program manager at Microsoft. Miller can be reached at wm@getwired.com.