Edit
  • Article

[Newsletters Archive ^] [< Volume 1, Number 2] [Volume 1, Number 4 >]

The Systems Internals Newsletter Volume 1, Number 3

http://www.sysinternals.com
Copyright (C) 1999 Mark Russinovich

June 19, 1999 - In this issue:

  1. WHAT'S NEW AT SYSTEMS INTERNALS

    • SDelete v1.1
    • Strings v2.0
    • LoggedOn
    • Filemon v4.13
    • DebugView/EE v3.1
    • "Inside NT Networking"
    • June "NT Internals"
    • NTFrob Update Status
    • Not-So-New Stuff

  2. INTERNALS NEWS

    • Numega DriverStudio Released
    • June Platform SDK Available
    • Win2K System File Protector (SFP)
    • Closing Files Opened from the Network

  3. WHAT'S COMING UP

    • An "AWE"-some Win2K API

SPONSOR: WINTERNALS SOFTWARE

The Systems Internals Newsletter is sponsored by Winternals Software, on the Web at http://www.winternals.com. Winternals Software is the leading developer and provider of advanced systems tools for Windows NT/2K. Winternals Software products include FAT32 for Windows NT 4.0, ERD Commander (boot-disk capability for Windows NT), and NTRecover.

Winternals Software announces the release of Regmon and Filemon Enterprise Editions. These utilities provide all the functionality of the freeware Filemon and Regmon, and add these powerful features:

  • view Registry and file system activity taking place on remote Win9x/NT systems
  • log output to a file in real-time
  • copy output lines to the clipboard
  • highlight lines that match a filter
  • view output from different computers simultaneously
  • print output directly to a printer
  • easily recall your last 5 filter selections

Get ordering and pricing information at http://www.winternals.com.

Hello everyone,

Welcome to the third edition of the Systems Internals newsletter. The newsletter currently has 4400 subscribers.

In the last newsletter I pointed out how Microsoft has done away with the Blue Screen of Death as we know it moving ahead to Windows 2000 (Win2K). The new Win2K Blue Screen does not have the loaded driver and stack dump information that is present on the Blue Screen's of prior versions of Windows NT. I asked if you find the information Microsoft has stripped useful and wish they had left things alone. The response was virtually unanimous, with every single respondent (except one) wondering why Microsoft is going for the lowest common denominator. Here is a typical opinion, sent in by Tony Lavinio:

So, in other words, this is the customer response that Microsoft is basing their decision on:

'I don't understand it, so it must be bad; make it go away.'

Why don't they just remove the whole screen, and put up a message saying "Pull Plug, Reinsert Plug, Start Over"? Why are they taking away one of the few clues we have as to why things went sour?

At least before, if it was a virus scanner or defragmenter or buggy driver, we'd have a point from which to start searching.

If this tool only helps 1 in 10,000 of us, with the broad base of NT deployment, it's worth it. Especially since we .01% support a good portion of the other 99.99%.

Who was the lone dissenter? It's not too surprising that it's someone from Microsoft that fields Blue Screen crash reports. Here's their slant on the change, which confirms Tony's speculation as to the reason for it:

I work in the NT Setup group in PSS at MS (which handles blue screen troubleshooting). I can assure you that the majority of people I talk to don't know what to do with the information on a 4.0 blue screen. I'm sure if you saw a stop 0xA with NAIFILTR.SYS all over the stack you'd know to update your antivirus, but most people don't make that connection, and really they don't realize that the stop code and params are useful to them. People who understand how to interpret blue screen data will probably be annoyed, but unfortunately they are a minority.

As I stated in the last newsletter, I feel that Microsoft should carry the NT 4.0 Blue Screen forward, keeping the loaded driver list and stack dump. I also think that they could make the Blue Screen better by providing more information, not less. For example, why not show the name of the currently executing process at the time of the crash? Or have more BugCheck calls pass the address of the faulter, not just the address that faulted? A major reason PSS ran into so many customers that are clueless about the Blue Screen is that Microsoft never wrote documentation on how to read it. At least part of the blame for user ignorance therefore rests on Microsoft's own shoulders.

If you want to know more about how Blue Screens occur and what's on the (NT 4.) Blue Screen, see my December 1997 article, "Inside the Blue Screen", from Windows NT Magazine (you can get to the on-line version from http://www.sysinternals.com/publ.htm).

As usual, please pass the newsletter on to friends that you think might find it interesting.

Thanks!

-Mark

WHAT'S NEW AT SYSTEMS INTERNALS

SDELETE V1.1

In the last newsletter I introduced SDelete, a secure delete utility that you can use to irretrievably destroy file data, as well as to clean a disk's free space of previously deleted data. The first version of SDelete was unable to securely overwrite the names of files you delete with it. A file's name often reveals sensitive information, and using SDelete to delete a file with a revealing name could leave that information exposed. In order to address this loophole I've updated SDelete to not only securely overwrite file data, but also file names. SDelete securely deletes a file name by renaming the file 26 times, replacing each letter in the file's name with successive letters of the alphabet, from 'A' to 'Z'.

Download SDelete v1.1 with full source code at http://www.sysinternals.com/sdelete.htm.

STRINGS V2.0

Executables and DLLs often have strings embedded in them that can reveal undocumented Registry values and error messages that hint at undocumented functionality. Unfortunately, most Windows NT/2K system DLLs and EXEs are written to use Unicode character strings, whereas traditional string searching tools like Grep only extract ASCII strings. I wrote the first version of my Strings utility a few years ago to scan binary files for ASCII or Unicode character strings. I've used it many times in my research of NT internals to help figure things out that Microsoft doesn't document.

Strings always had a major flaw, though, and that was its inability to take a wild-card expression as a file specifier so that you could scan multiple files in one shot. I wanted this feature so that, given a Registry value's name for example, I could easily determine which system DLLs reference it.

At last, I've updated Strings to take full wild-card file names, as well as to recurse directories. If you specify a wild-card expression Strings automatically prefixes output lines with the name of the file a string is found in, so you can do something like this:

strings *.dll | grep SafeBoot

Viewing the output of this expression (a version of Grep is available in the Windows NT Resource Kit Posix utilities) informs you of which system DLLs check the SafeBoot Registry key on Windows 2000. Strings is also extremely useful for seeing what new Registry values Win2K system DLLs, drivers and executables use. For instance, I used Strings to compare the Registry values referenced in the NT 4.0 SP4 TCP/IP stack (tcpip.sys) to those referenced by the Windows 2000 TCPIP stack. Here are about half the values that are new to Win2K's TCPIP stack (all of which I am assum are located under HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters):

ReservedPorts
DefaultGatewayMetric
InterfaceMetric
TempLeaseExpirationTime
TempIpAddress
TempMask
DhcpDefaultGateway
TcpWindowSize
TcpInitialRTT
TcpDelAckTicks
EnableTrafficControl
EnableTOSetting
MaxNormLookupMemory
MaxSendSegments
MaxFreeConnections
MaxFreeTWTcbs
FFPFastForwardingCacheSize

I won't hold my breath waiting for Microsoft to document all of the stack's new configuration parameters.

You can download Strings v2.0 at http://www.sysinternals.com/misc.htm.

LOGGEDON

Have you ever wanted to know who was logged on to a remote NT system? If so, then you'll want to download LoggedOn. LoggedOn is a simple utility that will tell you what users are interactively logged on to the local computer or a remote one, as well as what users are logged on via resource connections (e.g. a file or printer share). Here is sample LoggedOn output:

C:\>loggedon main

LoggedOn v1.0 - Logon Session Displayer
Copyright (C) 1999 Mark Russinovich
Systems Internals - http://www.sysinternals.com

Users logged on locally:
MAIN\Administrator

Users logged on via resource shares:
MAINDOM\MARK

Windows NT and Win2K don't provide an API that applications can use to determine who is logged on to a computer, but LoggedOn determines this by looking at the Registry keys that are loaded into a system's HKEY\USERS Registry tree. The profiles of any user that is logged on interactively are loaded into this key, and profiles have names that identify the SID (Security ID) of the profile's associated user account. For example, if you open Regedit and look under HKEY_USERS you will see something like:

HKEY_USERS\.DEFAULT\S-1-5-21-734676951-386466661-1233803906-500

Here, only one user is interactively logged on. You can tell its either the local or domain administrator because the RID (Relative ID) is 500, which is the RID NT reserves for administrator accounts.

By using APIs that permit one system to look into the Registry of another system, LoggedOn reads the HKEY_USERS key of a remote computer and converts the SIDs it finds into account names. To determine who is logged on via resource share LoggedOn uses the NET API, which is documented in the SDK. The Net command-line tool makes extensive use of the NET API. A side effect of LoggedOn accessing a remote system's Registry is that the account you run LoggedOn in will always appear as logged on to a remote system via a resource share in LoggedOn's output.

You can download LoggedOn with full source code at http://www.sysinternals.com/misc.htm.

FILEMON V4.13

Filemon v4.13 has just been released, an update that reflects changes to the Windows NT filer driver and corrects a bug I inadvertently introduced into the 4.12 driver. The 4.13 filter driver has these changes:

  • it uses the Resource synchronization type to protect some of its internal data structures
  • it handles the new Win2K IRP, IRP_MJ_PNP_POWER

The Resource synchronization type is virtually undocumented in both the Windows NT 4.0 and Win2K Device Driver Development Kits (DDKs). The Design Guide does not even mention Resources, while their functions are documented in the Reference under the "Executive Support Routines" section. Resources are a useful mechanism for protecting data structures that can be read simultaneously by different threads, but that require exclusive access by a thread during an update. Thus, they are reader/writer locks that are acquired for shared access by readers and exclusive access by writers. File system drivers make extensive use of Resources, so I felt it appropriate to update Filemon to use them where appropriate.

Filemon v4.13 also handles the new IRP_MJ_PNP_POWER IRP so that it is a power and plug-and-play friendly filter driver when it runs under Win2K. The only requirement of a file system filter driver in handling IRPs of this type is to pass them to the file system devices the filter is attached to.

You can download Filemon v4.13 with full source code at http://www.sysinternals.com/filemon.htm.

DEBUGVIEW/EE V3.1

DebugView/EE is a versatile debug output monitor that you can use to capture local or remote debug output generated by Win32 programs or kernel-mode device drivers under Win95, Win98, WinNT, and Win2K. Its usefulness is limited in environments where a user experiences a crash using a device driver, however - all the debug output DebugView captures before a crash is lost. The latest version of DebugView/EE addresses this problem on Windows NT/2K. If a user is capturing kernel-mode output generated by a device driver and the user has configured NT to save a crash dump, then DebugView/EE can extract the debug output from the dump file when the system reboots. Use DebugView/EE's Edit|Process Crash Dump menu selection to have it scan a memory dump for debug output. This feature enables users to send you back a text file containing the debug output your driver generated right up to the time of the crash.

Download DebugView/EE v3.1 at http://www.sysinternals.com/debugview.htm.

"INSIDE NT NETWORKING"

My March 1999 Windows NT Magazine "NT Internals" column is now on-line. Learn about NT's networking architecture from top to bottom, including what APIs it implements, how protocol stacks interface with APIs, and how hardware vendors write network drivers to work with protocol drivers. Also, find out about some new features of Win2K's networking, including deserialized NDIS and ATM support.

Read "Inside NT Networking" and other past NT Internals columns on-line at http://www.sysinternals.com/publ.htm.

JUNE "NT INTERNALS"

The June installment of my Windows NT Magazine column is "Inside EFS, Part 1". This article describes the architecture of Microsoft's Encrypting File System (EFS), and takes you on a detailed walk through the steps EFS follows when it encrypts a file. EFS provides a transparent encryption facility for Win2K NTFS drives and Microsoft developed it specifically to address our NTFSDOS tool's ability to read NTFS files without regard to their security. This column will be available on-line in three months.

Two newsletters ago I talked about how the EFS APIs necessary to backup and restore encrypted files are undocumented. Unfortunately, these APIs are still not documented in the current edition of MSDN. However, I have been informed that Microsoft is sending the documentation, which is marked as "Microsoft Confidential", to its partners and to backup software vendors. Also, David Golds, File Systems Program Manager at Microsoft, presented a talk on file system enhancements for Win2K at the recent Microsoft TechEd conference in Dallas. In the presentation, the slides for which you can view on-line at http://www.teched99.com/slides/1-337.ppt, he mentions that the backup APIs are undocumented but that you can bug him of you want the documentation. Unfortunately, his e-mail address is not listed on the slides.

Visit http://www.winntmag.com for Windows NT Magazine subscription information.

NTFROB UPDATE STATUS

NTFrob is a utility I released a couple of years ago that allows you to precisely configure the quantum lengths of foreground and background processes on NT 4.0. NTFrob modifies data structures internal to the NT kernel, so it is highly Service Pack-specific. Since the release of NT 4.0 Service Pack 5 I have been deluged with queries asking when I'm going to release NTFrob v1.5, the SP 5 update. The answer is that the update is forthcoming - I'm waiting for Microsoft to provide MSDN subscribers with SP5, including debug information. I require debug information to update NTFrob for new Service Packs.

You can download NTFrob for NT 4 SP0-4 at http://www.sysinternals.com/ntfrob.htm.

NOT-SO-NEW STUFF

A few months ago I released a Win9x/NT/2K serial and parallel port monitor at Systems Internals. Portmon allows you to see exactly how applications interact with serial and parallel ports, including what data they send and receive. You can use it to watch dial-up sessions, laplink serial connections or printer activity. Portmon has been a huge hit, and it recently garnered 5 stars from Ziff-Davis' Software Library, the highest rating possible. Other Systems Internals tools that have earned 5-stars include Regmon, NTFSDOS, and BlueSave.

Download Portmon at http://www.sysinternals.com/portmon.htm.

INTERNALS NEWS

DRIVERSTUDIO RELEASED

CompuWare's NuMega Labs has released DriverStudio, a comprehensive toolkit for Windows 9x/NT/2K device driver developers. It includes SoftICE 4.0, BoundsChecker for Drivers, VtoolsD, DriverAgent, DriverWorks, FieldAgent for Drivers, and in the future will add TrueTime for Drivers and TrueCoverage for Drivers. Like I said in the last newsletter, this is a must-have developer toolkit. NuMega has also launched a device driver developer-oriented web site called "Driver Central" - http://www.numega.com/drivercentral/default.asp.

JUNE PLATFORM SDK RELEASED

You can download the June release of Microsoft's Platform SDK now at http://www.msdn.microsoft.com/developer/sdk/platform.asp.

WIN2K SYSTEM FILE PROTECTOR (SFP)

One of the biggest gripes of NT systems administrators and users is NT's "DLL Hell". DLL hell is the result of many applications updating key system DLLs with versions that they bundle. Applications typically do this so that they can guarantee that they work properly, however, when they replace a DLL they many times break other applications by installing incompatible versions, or even "update" a DLL to an older version.

Microsoft has addressed DLL versioning problems in Win2K with the introduction of the System File Protector (SFP). Actually, its name will change soon to Windows File Protector (WFP), but as of Beta 3 (build 2031) it's still SFP. SFP is implemented in a DLL named sfc.dll that the Winlogon process (winlogon.exe) loads when the system boots. SFP includes a built-in list of about 3000 standard Win2K system DLLs, executables (.exe), installation files (.inf), drivers (.sys) and font (.fon) files that are installed in 30-40 different directories. When SFP initializes it executes a change-notify directory operation on each directory that contains files that it protects. When it detects tampering with a file it pops up a dialog box informing the current user, writes an even to the event log, and replaces the modified file with a backup stored in %systemroot%\system32\dllcache. If the backup file that SFP looks for in dllcache is missing or has also been tampered with, SFP retrieves a fresh copy from the Win2K installation media.

To see what files SFP protects you can use the Strings utility mentioned elsewhere in this newsletter to dump the Unicode string names embedded in %systemroot%\system32\sfc.dll.

The only utilities that can update system files are hotfix.exe, service packs (update.exe), upgrade installations, and the Win2K Update service. How do these tools bypass the SFP? They temporarily disable it by calling the exported sfc.dll function SfcTerminateWatcherThread, and they make sure to reflect updates in the dllcache subdirectory. You should note that Win2K requires all system files to be digitally signed by Microsoft, so it's generally not possible to update a system file with an arbitrary version of your own.

Win32 programs can watch for changes in a directory by using the FindFirstChangeNotification and FindNextChangeNotification Win32 APIs. However, these APIs simply inform an application that something has changed; they do not tell the application exactly what has changed. An application is therefore required to scan an entire directory to determine which files or subdirectories may have changed. SFP uses the NT Native API to perform a change notification request where NT tells it exactly what files or subdirectories change in monitored directories. The function SFP uses is named NtNotifyChangeDirectoryFile, and, like 90% of NT's Native API, it is undocumented. Look for an applet at Systems Internals in the near future that shows you how to use NtNotifyChangeDirectoryFile.

My September "NT Internals" column, "Inside Win2K Reliability Enhancements, Part 2" describes SFP in more detail.

CLOSING FILES OPENED FROM THE NETWORK

One of the most frequent questions I get from Systems Internals visitors is "how do I close files that users have open from the network?" If a user has a file or directory remotely opened, then it's not possible for you to delete, rename or update the file or directory locally. A similar question is, "How do I see what files users have opened from the network?" Both of these questions are answered with the Net command-line utility that comes with Windows NT/2K. To see what files are opened simply type "net file". You'll get a listing of open file names, corresponding file name identifiers, and the names of the users that have files opened. To close one of the files you see open you type net file <id> /close. To view the files opened locally you can use my NTHandle or HandleEx tools.

The APIs that underlie the Net command's file viewing and closing functionality are documented in the Platform SDK and in the MSDN Library. Use the NetFileEnum API to enumerate open files and the NetFileClose API to close an open file. The APIs actually let you enumerate the open files on remote servers, something that the Net command doesn't allow.

NTHandle is available at http://www.sysinternals.com/nthandle.htm. HandleEx is available at http://www.sysinternals.com/handleex.htm.

WHAT'S COMING UP

AN "AWE"-SOME WIN2K API

Win2K introduces a new API called AWE (Address Window Extensions) that memory-intensive applications can use to directly access and manage large amounts of physical RAM - even more than 3GB, the upper-limit of RAM that a Windows NT application can address in its virtual address space. In fact, if a x86 system has PSE (Page Size Extensions) and more than 4GB of RAM, an application can use AWE to take advantage of all of the computer's memory. This API is therefore ideal for memory hungry applications like Web servers and database servers. Next time I'll tell you how to use the APIs, both from Win32 applications and from device drivers.

While I'm on the subject of memory-hungry applications, here's a tip for anyone writing an application that caches files (like a Web Server). The Windows NT Cache Manager divides its Cache memory into 256KB slots called "views". If a file that is less than 256KB in size is cached the Cache Manager must still assign the file an entire 256KB slot, which means that part of the Cache's virtual memory is wasted. Thus, it is generally more performance efficient to cache files less than 256KB in size in your own application's virtual memory, and to rely on the file system to cache files greater than 256KB in size. IIS 5.0 uses this trick.

Thank you for reading the Systems Internals Newsletter.

Published Saturday, June 19, 1999 7:14 PM by ottoh

[Newsletters Archive ^] [< Volume 1, Number 2] [Volume 1, Number 4 >]