he Microsoft® .NET Framework allows you to develop robust systems and applications that were formerly impossible using Visual Basic® alone. With Visual Basic .NET and the classes in the .NET Framework, you can write applications requiring multithreading and thread pooling. You can use the classes in the System.Diagnostics namespace to trace and record events during troubleshooting. Windows® services can now be built with Visual Basic .NET via the System.ServiceProcess.ServiceBase class. Another thing you can do is file system monitoring using the System.IO.FileSystemWatcher class. These are a powerful set of features from which to build an application. I'll discuss each of these features in some detail then tie them together into a sample multithreaded Windows service that imports XML data files into a SQL Server™ database.
Tracing and Error Logging
When building a robust application, you must pay careful attention to diagnostics and troubleshooting mechanisms, both before and after it's been deployed. This is especially true for complex server apps where problems may not be easily reproducible. In situations like this, you typically write a tracing component, which handles opening the output destination (event log or file), writing the trace message, and closing the output destination, and then you call methods on this component throughout your code.
In most cases, you don't want to output trace messages all the time, but rather only when you're diagnosing a problem, so you define levels or severities for messages, such as Tracing, Warning, and Error. When you call the logging component, you specify the message severity; the component determines whether to log the message based on the current system tracing level setting (which may be a registry setting). Tracing may also require registering the event source and creating a message resource file for logging.
Tracing with .NET
The .NET Framework includes classes and data types that make it easy to log trace messages—the logging infrastructure is right there for you. Figure 1 lists the .NET Framework classes that you use for tracing. The classes are part of the System.Diagnostics namespace. The Trace class exposes numerous static (shared) methods. The Write methods let you log a specific message, while Assert lets you log a message if a specified condition is false. The Fail method is similar to an Assert; the condition is always false, so the Fail method always logs the specified message.
The Trace class outputs your messages to objects in its Listeners collection. The framework includes EventLogTraceListener, which writes to an event log; TextWriterTraceListener, which writes to a text file; and DefaultTraceListener, an instance of which is added by default to the Trace class's Listeners collection. The DefaultTraceListener class's Write and WriteLine methods emit the message to the attached debugger, which may or may not display the message.
You can implement your own trace listener. You might want to receive trace output from your application running on a remote machine behind a firewall. In that case, you can write a trace listener that sends trace messages to your server over HTTP requests. Of course, this would significantly degrade your application's performance, but only while tracing is enabled.
While the Write and WriteLine methods provide the basic functionality you need to log trace messages, WriteIf and WriteLineIf let you control whether a message is to be logged based on the application's trace level setting. There are three ways that you can control tracing. You can define the compilation constant TRACE and set it to True, telling the compiler to keep your tracing code (for example, calls to Trace.WriteLine) in the compiled binary. You can do this using the compiler command-line parameter /d:TRACE=True or by checking the Define TRACE constant box in the Visual Studio project properties dialog, as shown in Figure 2. Setting this constant to False causes the compiler to strip off all tracing code from the compiled binary.
Figure 2 Enabling Tracing in Visual Studio .NET
Figure 3 shows the Microsoft intermediate language (MSIL) output for a method that was compiled with TRACE=True. You'll notice there's a call to System.Diagnostics.Trace::WriteLineIf. Compare that to Figure 4, which is the MSIL output for the same method compiled with TRACE=False, which causes all calls to the Trace class methods to be removed by the compiler. The TRACE compilation constant is an effective way to make debug or troubleshooting versions of your programs with lots of tracing code; to create a more efficient production version, you just have to change a single constant. What's more, you don't have to enclose all tracing calls in #If statements as you would in Visual Basic 6.0.
The second way you can control tracing during execution of an app without having to recompile is using the BooleanSwitch class. You do this by specifying whether tracing is on or off in a registry value or in an environment variable. BooleanSwitch's constructor accepts two parameters: the switch's name and its description.
Private Shared BITraceBooleanSwitch As New _
"Batch Importer Trace Control")
The switch is off by default. To turn it on, either set a registry value or an environment variable. The registry value for this particular switch is:
Note that the value name matches the switch's name that you specify when instantiating the switch object. You set this value to 0 or 1 to turn the switch off or on, respectively. You can also set the environment variable _switch _BITraceControl to 0 or 1.
This switch has no effect until you use it in your own If...Then statement or with Trace.WriteLineIf. For example:
"this means the switch is" &_
This line will log the message "this means the switch is turned on" only if the switch is turned on. Otherwise, it'll simply ignore it. The switch value is read once during execution (an optimization to avoid repetitive reading from the registry), so if you change the registry setting, you'll need to restart the application for the change to take effect.
In most cases you'll have various types of trace messages like error, warning, and information messages. Use TraceSwitch to control which message levels are logged. Like the BooleanSwitch, you specify TraceSwitch's name when you instantiate it and you set its value in the registry or an environment variable. The possible values for a TraceSwitch are 0 (off), 1 (errors), 2 (warnings), 3 (information), and 4 (verbose).
TraceSwitch has four properties: TraceError, TraceInfo, TraceWarning, and TraceVerbose. Each of these properties returns True if messages of that type are allowed to be logged. If you set the tracing level to 3 (information) then TraceError, TraceWarning, and TraceInfo will all return True, while Trace Verbose will return False. You can use TraceSwitch with Trace.WriteLineIf like this:
"This is my informational message")
BITraceLevel is an object that is instantiated from TraceSwitch. Figure 5 demonstrates using Trace, EventLogListener, and TraceSwitch to write messages to the event log. Messages will appear in the application log on the local computer with the message source set to BatchImporter.
Creating and Controlling Threads
Now let's turn to the subject of threads. One of the great features of the .NET architecture is the ability to create multithreaded applications in Visual Basic without having to use third-party tools or unsupported tricks in Visual Basic. .NET's multithreading support is provided by classes and interfaces in the System.Threading namespace. The core class is System.Threading.Thread, and it is what you use for creating and controlling threads. To create a thread, create a new System.Threading.Thread object, passing the constructor a ThreadStart delegate. This delegate represents the method where the thread will begin its execution:
Public Delegate Sub ThreadStart()
As you can see, the function you create as the thread's starting point accepts no parameters. But you often want to pass some parameters to the new thread. Do this by setting properties on the object that expose the thread start function. For example, the code in Figure 6 instantiates an object called fp from the FileProcessor class and then sets its FilePath property. It then creates a new thread using the FileProcessor.ProcessFile as the thread start function.
After creating the thread, you start it by simply calling the asynchronous method Thread.Start. This method returns immediately, possibly before the thread has really started. You can use Thread.ThreadState or Thread.IsAlive to determine whether the thread has actually started running. For example, the code in Figure 6 uses ThreadState to determine the thread's current state and write it to the console. If the thread has not yet started, its state will be Unstarted. Notice the use of Thread.Join to wait until the thread has actually died.
When the worker thread does start, it will write a line to the console informing the user that it's about to go to sleep. The worker thread then calls the shared (static) method Thread.Sleep, which puts the thread in a Wait SleepJoin state for the specified period of time, simulating a long-running operation. Meanwhile, the main thread is in a while loop where it puts itself to sleep to give the worker thread some time to output its message and go to sleep.
When the main thread wakes up it checks if the worker thread has entered the WaitSleepJoin state and if it has, the main thread then prompts the user to enter "i" to interrupt the worker thread or "a" to abort it.
Normally, you call Thread.Stop to stop a thread that's currently running. If the thread is in WaitSleepJoin state like this worker thread, you can get its attention by calling Thread.Interrupt, which throws the ThreadInterruptedException in the thread. You can also abort a thread using Thread.Abort even if it's in the Wait SleepJoin state.
Aborting a thread throws the ThreadAbortException. According to the .NET documentation, this exception is not catchable, so it causes the thread to terminate (after executing the Finally block). However, in Beta 1, this exception is caught and the Catch block is executed. In my example, the worker thread catches this exception, then exits.
When you kill a thread, you want to wait to make sure it's dead. The thread can take a while to die if, say, it is executing lengthy code in its Finally block. You use Thread.Join to wait for a thread to stop execution.
A stopped thread cannot be restarted; you must start a new thread. If you just want to pause a thread that's in the Running state for a while with the intent of resuming it later, use Thread.Suspend, then Thread.Resume.
A multithreaded application usually has resources that can be accessed from multiple threads; for example, a global variable that is incremented or decremented by multiple threads. It is sometimes desirable to prevent multiple threads from concurrently altering the state of a resource. The .NET Framework includes several classes and data types that you can use to synchronize actions performed by two threads.
The simplest case is if you have a shared variable that you need to update from different threads. To do this, you can use the System.Threading.Interlocked class. For example, to increment or decrement the shared variable called num, you'd write Interlocked.Increment(num) or Interlocked.Decrement(num). You can also use Interlocked to set the variables to a specific value or to check the equality of two variables.
If there's a section of code in an object's method that should not be accessed concurrently by multiple threads, you can use the Monitor class to acquire a lock on that object by calling Monitor.Enter(object). Any other thread wanting to execute the same code would need to acquire the same lock and will be paused until the first thread releases the lock by calling Monitor. Exit(object).
For more control over thread synchronization or for cross-process synchronization, use the Mutex class, which is a named synchronization object that can be obtained from any thread in any process. Once you create or obtain the mutex, you use its GetHandle method to (as you'd expect) get a handle that you can use with the WaitHandle.WaitAny or WaitHandle.WaitAll methods. These two methods are blocking and will return only if the specified handle is signaled (that is, the mutex is not being used by another thread) or if the specified timeout expires. After you obtain the mutex, you perform the necessary synchronized processing and then call Mutex.ReleaseMutex to release it.
Sometimes you need a mechanism for one thread to notify other threads of some interesting event that occurred. In those cases you can use the .NET synchronization event classes, ManualResetEvent and AutoResetEvent. In the world of thread synchronization, an event is an object that has two states: signaled and nonsignaled. An event has a handle and can be used with WaitHandle just like a mutex. A thread that waits for an event will be blocked until another thread signals the event by calling ManualResetEvent.Set or AutoResetEvent.Set. If you are using a ManualResetEvent, you must call its Reset method to put it back to the nonsignaled state. An AutoResetEvent will automatically go back to nonsignaled as soon as a waiting thread is notified that the event became signaled.
The .NET Framework provides a convenient mechanism for queueing work to be performed by one of a pool of threads. This is common in server applications that handle multiple concurrent work items or work requests. My sample application waits for input files and imports them into a database. The System.Threading.ThreadPool lets you queue work using the shared method, QueueUserWorkItem, which comes in two overloaded versions. Both versions accept a WaitCallBack delegate that represents the function to call when a thread is dispatched from the pool to perform this work item. The second QueueUserWorkItem version also accepts an object that is passed to the WaitCallBack delegate when the work item executes. You use this object as a container for state information that you want to pass to the delegate function.
Dim fileinfo As New FileInformation(e.FullPath)
Dim worker As New Importer()
New WaitCallback(AddressOf worker.ProcessFile), fileinfo)
Here, fileinfo contains some state information and is passed as the object parameter to the delegate function worker.ProcessFile.
Note that the thread pool automatically manages the creation of threads in the pool. You have no control over the maximum or current number of threads in the pool. It is all determined using heuristics that take into account available memory, number of processors, and number of threads currently running. Because threads in the pool are completely controlled by the system, your WaitCallback functions should not stop the current thread in use (for example, by calling Thread.CurrentThread.Abort) because this would kill a thread that belongs to the pool.
Creating Windows Services
In the past, if you wanted to create a Windows service, you did it using Visual C++®. You couldn't do it properly in Visual Basic. Sure, you can use utilities like srvany or third-party utilities that let you create services in Visual Basic, but the end result is usually a hack that just doesn't feel right. I was thrilled to find out that Visual Basic .NET lets you create Windows services using the .NET Framework classes.
Writing a service from scratch using C++ or C involves quite a bit of coding for communication between your service and the Service Control Manager (SCM), which handles starting, pausing, continuing, and stopping services. The .NET Framework makes it easy to implement a service by providing the System.ServiceProcess.ServiceBase class. When you write a service, you inherit from this class and override some of its methods and set its properties (see Figure 7) and you're ready to go. It's that easy.
Figure 8 shows a sample service implemented by a class named FileMonitorService. This class inherits from ServiceBase and overrides some of its methods to control starting, pausing, continuing, and stopping. FileMonitorService's constructor sets the service's CanPauseAndContinue and ServiceName properties. The class overrides ServiceBase.OnStart to actually get the service started. In the OnStart override, it writes a trace message to the event log and then calls StartMonitorThread. This method creates a worker thread and uses it to do the actual processing. OnStop logs a trace message and calls StopThread, which interrupts and then kills the thread started by StartMonitorThread. When the service is paused, OnPause logs a message and then stops the worker thread. When the service resumes, OnContinue creates a new worker thread.
To run this service, your Main procedure instantiates an object from FileMonitorService and passes it to ServiceBase.Run:
Public Sub Main()
Dim BIService As New FileMonitorService()
BIService.AutoLog = False
You can control whether the service control events are automatically logged by setting the service's AutoLog property to True or False. If you set it to True, the ServiceName property is used as the message source for messages logged to the event log. So be sure to set ServiceName before you set AutoLog to True, or you'll get an exception. You can also set the object's AutoLog property to False to disable automatic logging of service control events.
After creating the service, you'll want to install and run it. If you started with a Visual Studio Windows Service project, you can select the service component and click on the Add Installer hyperlink in the Properties window. Visual Studio will add an installer component to your project with the necessary code to install the service and register the event log source.
Alternatively, you can add a module to your project and insert it into the code shown in Figure 9. This code demonstrates how to create a class that inherits from System.Configuration.Install.Installer, adds to it the RunInstaller attribute set to True, and uses System.ServiceProcess.ServiceInstaller and System.ServiceProcess.ServiceProcessInstaller to specify the service name and the startup mode.
Once you've added this installer component, you can build your service executable. You then install it from a command prompt using InstallUtil.exe:
This will instantiate and call the installer component that you added to your project. If you want to uninstall the service, you can do so using InstallUtil with the /u command-line option:
InstallUtil /u BatchImporter.exe
Monitoring the File System
The last concept I'll cover is using the .NET Framework's built-in support for file monitoring. You may have an application that needs to wait and process files that show up in a particular directory; for example, an application that imports data from a file into a database. Data files may be downloaded from a mainframe or otherwise transferred into an input directory where the application imports them into a database. Instead of constantly polling the directory for new files, you can wait for notifications that a new file has been created. You can do this in Visual Basic 6.0 using Win32® APIs. You can do this in Visual Basic .NET using the .NET Framework classes. I can't say it's easier with .NET (it wasn't that hard in Visual Basic 6.0 to begin with), but it is definitely a more object-oriented way to monitor the file system than using the Win32 API. That does not automatically make it better or worse, just different and more consistent with how you write applications on the .NET platform.
The class you use for watching the file system is the cleverly named System.IO.FileSystemWatcher. Figure 10 shows this class watching for .xml files being created in the specified path. First, instantiate an object from this class and then set its Path property to the directory you want to watch. You then use a variety of properties to specify what changes you are interested in. I used the Target property to specify that it should watch only for changes in files and not subdirectories. So if someone renames a subdirectory, my application will not be notified, which is what I want.
The IncludeSubdirectories property lets you specify whether the watcher should also watch subdirectories under the directory specified by the Path property. Setting this property to True causes the watcher to watch every subdirectory recursively. Do not confuse this with the Target setting. You can be watching for changes only in files (Target=IO.WatcherTarget.File), but still be interested in changes in files that occur in any of the subdirectories. In this example I am only interested in watching the directory specified in the Path property, so I set IncludeSubdirectories to False.
You can also use the Filter property to specify the file name to watch. Here I use "*.xml" to indicate I'm interested only in .xml files. Furthermore, if you are interested in changes to files, you can indicate the specific types of changes to watch for by setting the ChangedFilter property to Attributes, LastAccess, LastWrite, Security, or Size. In this example I am only interested in new files being created (rather than changes to existing files) so I don't set the ChangedFilter property. You should try to be as specific as possible about what to watch and what changes you're interested in. This helps reduce the number of notifications your application receives, which makes it less likely that the buffer used to hold these notifications will overflow and lose notifications.
Next, hook up event handlers for the various events you're interested in. FileSystemWatcher events are Changed, Created, Deleted, Error, and Renamed. In order to handle an event, you must first write an event handler with the same declaration as the FileSystemEventHandler delegate:
Public Delegate Sub FileSystemEventHandler( _
ByVal sender As Object, _
ByVal e As FileSystemEventArgs )
For example, at the end of Figure 10 you see OnFileCreated, which has the same declaration as the FileSystemEventHandler delegate that was just shown. Before you begin waiting for change notification, you should hook up the event handler using AddHandler and the AddressOf operator:
AddHandler fw.Created, New FileSystemEventHandler( _
Now you're ready to start waiting for notifications. Set the Enabled property to True and then call WaitForChanged, which takes an argument that indicates the type of changes to wait for. In this example, I specify that I want to wait only for Created changes; that is, when a file is created. There's an overloaded WaitForChanged method that takes a second argument specifying the time in milliseconds to wait.
When a change occurs (or after the timeout expires), WaitForChanged returns a WaitForChangedResult structure, which contains four members. WaitForChangedResult.ChangeType lets you know the kind of change that occurred, which is useful if you are waiting for multiple types of changes. If the timeout expires, WaitForChangedResult.TimedOut is set to True. WaitForChangedResult.Name is the name of the file (or subdirectory) that has changed. If that change was a file or subdirectory being renamed, then WaitForChangedResult.OldName contains the original name.
A word of caution about waiting for files to be created. You receive the notification immediately when the file is created, which means the application that created the file may still have it open and be writing to it. For example, if an application is depositing a large file into your input directory via FTP, it may take several seconds or minutes to write the entire file. If you attempt to process the file as soon as you receive the Created notification, you'll be processing an incomplete file that is being written to, something you almost certainly want to avoid. As a workaround, you can attempt to open the file exclusively when you receive the notification (see Figure 11). If this fails because the file is still open by another application, you can wait for a period of time and retry. If you are dealing with huge files that take hours to write out to disk, you may want to make a list of all files for which you receive Created notifications and process them in batch at a later time.
Note that FileSystemWatcher notifications are dispatched on threads from the thread pool, so make sure the callbacks you use to handle these notifications are thread-pool-safe. That is, they should not use thread-local storage or otherwise expect to always execute on the same thread.
Putting it Together
The sample Visual Basic .NET application I've written is a Windows service that waits for XML data files to be deposited in an input directory and then imports them to a SQL Server database. The service is multithreaded. There's the service main thread, the worker thread that monitors the input directory, and the thread pool that's used to process input files.
Figure 12 shows the sequence for starting the service, with the corresponding code shown in Figure 13. The Main procedure behaves differently in debug and release builds. In debug builds, it calls FileMonitorService.StartMonitorThread directly. In release builds, it calls ServiceBase.Run, which causes FileMonitorService.OnStart to be called. ServiceBase does not directly call FileMonitorService.OnStart; it was just convenient to diagram it that way.
Within OnStart, StartMonitorThread is called, which instantiates an object from the MonitorFiles class and creates a new Thread using MonitorFiles.StartMonitor as the thread start function. Finally, StartMonitor calls Thread.Start to start the new thread and returns. As far as the SCM is concerned, the service is now started. As you can see in Figure 12, StartMonitor and WaitForChanged are called on the worker thread, not the main service thread.
The new thread starts and calls MonitorFiles.StartMonitor. This procedure creates a FileSystemWatcher object and uses it to monitor the input directory (indicated by the constant dirpath), specifying MonitorFiles.OnFileCreated as the callback function for file creation events. It then calls FileSystemWatcher.WaitForChanged to wait for new files being created in the input directory. At this point, the worker thread is blocked in a WaitSleepJoin state, waiting for notifications from the FileSystemWatcher.
When a file is created in the input directory, the FileSystemWatcher notifies the service by calling MonitorFiles.OnFileCreated, which queues a new work item on the thread pool. Figure 14 shows the sequence, and Figure 15 shows the relevant code.
Figure 14 Handling File Creation Notifications
OnFileCreated uses ThreadPool.QueueUserWorkItem to queue a call to Importer.ProcessFile. This is the procedure that opens the input file and imports its contents into the database. The parameter to this procedure is an object of type FileInformation. This class acts as a state container for the file path and a sequential file number. The FileInformation constructor keeps track of the sequential file number by incrementing the member num. Because num is shared, there's a chance it'll be accessed concurrently from different threads, so I use System.Threading.Interlocked.Increment to increment its value.
Importer.ProcessFile first ensures that the input file is not open by another process by trying to get exclusive access to it using the technique I just mentioned. It then calls ImportData, which loads the file into a System.Xml.XmlDocument, and uses a System.Xml.DocumentNavigator to iterate through the order elements in the file, reading in the various order information. After reading each order, ImportData calls InsertOrder to create the INSERT statement and execute it using System.Data.SQL.SQLCommand (see Figure 16 and Figure 17). First my code gets exclusive access to the file and then opens it using a DOM document. For each <order> in the file, I create and execute an INSERT statement to insert the order data in a SQL Server database.
You've seen how to use the .NET Framework to build a multithreaded Windows service with Visual Basic. Your next step should be to learn more about the .NET class library, as it is the cornerstone of .NET development.