Exchange Server 2003 On-Demand Tasks
Microsoft® Exchange Server databases can grow over time and become fragmented, which can cause performance issues that may affect users. You can identify discrepancies in Exchange Server databases by using tools available in Exchange Server. You can also examine the Queue Viewer and monitor the performance of your Exchange servers and identify normal and abnormal trends. Typically, on-demand tasks are performed when errors are reported by monitoring tools, or by users reporting issues to your Help desk.
You may need to perform the following on-demand maintenance tasks:
Defragment mailbox and public folder stores Exchange databases can become fragmented over time which can cause performance problems. By defragmenting the databases, you can reduce the file size and create contiguous storage space. Defragment Exchange databases by using Exchange Server Database Utilities (Eseutil.exe).
Verify mailbox and public folder store integrity You can resolve inconsistencies in Exchange databases by verifying and repairing the integrity of the database using Information Store Integrity Checker (Isinteg.exe). You can also use the Eseutil.exe tool to check database integrity.
Examining queues You can use Queue Viewer to examine queues. This activity lets you identify normal and abnormal trends for messages that are visible in the queues. Many messages backed up in the queue can indicate a security threat, a spam attack, or a network performance issue.
Configure Performance console You can configure the System Monitor (Performance Monitor in Microsoft Windows NT® 4.0) to monitor the performance of your Exchange servers, so that you can establish what is normal and what changes you can make to improve performance.
Online and Offline Defragmentation of Exchange 2000 Server and Exchange Server 2003 Databases
Exchange Server databases require defragmentation. Specifically, Exchange Server database defragmentation refers to rearranging mailbox store and public folder store data to fill database pages more efficiently, which eliminates unused storage space. There are two types of Exchange database defragmentation: online and offline.
Online Defragmentation
Online defragmentation is one of several database-related processes that occur during Exchange database maintenance. By default, Exchange servers automatically run Exchange Server database maintenance between 01:00 (1:00 A.M.) and 05:00 (5:00 A.M.) local time, every day. Online defragmentation occurs while the Exchange Server databases remain online. Therefore, your e-mail users have complete access to mailbox data during the online defragmentation process.
The online defragmentation process involves automatically detecting and deleting objects that are no longer being used. This process provides more database space without actually changing the file size of the databases that are being defragmented.
Note
To increase the efficiency of defragmentation and backup processes, schedule your maintenance processes and backup operations to run at different times. Running the online backup after the online defragmentation has started will end the online defragmentation process at that time.
You can schedule database defragmentation in two ways:
To schedule database defragmentation for an individual database, use the Maintenance interval option on the Database tab of a mailbox store or public folder store object.
To schedule database defragmentation for a collection of mailbox stores and public folder stores, use the Maintenance interval option on the Database (Policy) tab of a mailbox store or a public folder store policy.
For more information about how to create a mailbox store policy or public folder policy, see "Create a Mailbox Store Policy" and "Create a Public Folder Store Policy" in Exchange 2000 Server or Exchange Server 2003 Help.
Offline Defragmentation
Offline defragmentation involves using Exchange Server Database Utilities (Eseutil.exe). Eseutil is an Exchange Server tool that you can use to defragment, repair, and check the integrity of Exchange Server databases.
By default, Eseutil is located in the <drive>:\<installation root>\exchsrvr\bin directory after running Exchange 2000 Server or Exchange Server 2003 Setup (where <drive> is the drive letter of the drive and <installation root> is the installation path where you installed Exchange Server).
You can perform offline defragmentation only when your Exchange Server databases are offline. Therefore, your e-mail users will not have access to mailbox data during the offline defragmentation processes. The database has to be in the "clean shutdown" state for offline defragmentation to run on it.
During the offline defragmentation process, Eseutil.exe creates a new database. It copies only the in-use database records to the new database file, which results in a new compact database file. An offline defragmentation is the only method that reduces the physical file size of the databases. You should perform an offline defragmentation in the following situations:
After performing a database repair (using the command Eseutil /p)
After moving a significant amount of data from an Exchange Server database
If instructed to do this when you are working with Microsoft Product Support Services, or when troubleshooting a specific problem and the existing documentation calls for an offline defragmentation.
Important
You should consider an offline defragmentation only if many users are moved from an Exchange Server database or after a database repair. Performing offline defragmentation when it is not required could affect performance. To determine how much space you will regain after the offline defragmentation of the database, check event 1221 in the Exchange server’s Application log. You should also consider the time factor when performing an offline defragmentation of the database because it is a lengthy process.
Important
It is also important to note that the offline defragmentation requires about 110% of the space of the original database to succeed. This is because the Eseutil tool actually creates a new database file, in addition to the original database file. Both files have to coexist on the disk. It is possible however, to redirect the temporary database file to a different hard disk by using the Eseutil /t switch. Using this switch increases the time required to complete the defragmentation process. You can also use a network disk. However, using a network disk is not recommended because it significantly increases the time required to complete the defragmentation process. Also, you are subject to the risk of network availability issues.
When you use Eseutil.exe to defragment your Exchange Server databases, consider the following:
When you run Eseutil.exe in defragmentation mode (using the command Eseutil /d), you can also include a /p switch. Including the additional /p switch during a defragmentation operation preserves the original database being defragmented in case you need to revert to this database. If the original database and the temporary databases are on different drives during the offline defragmentation process, using the /p switch bypasses the need to copy the defragmented database to the location of the original database. Then you need to manually rename the newly created temporary database using the same database name that the original database had and manually move the database to the correct location using Windows Explorer.
Because offline defragmentation actually creates the new database, database files, and log file signatures, you should create new backups of Exchange Server 2003 databases immediately after offline defragmentation. A new defragmented database file will have a different database signature on it. Because the databases and transaction logs point to each other based on signatures, all previous backups of this database are invalidated by the offline defragmentation. Table 1 provides information about the modes of operation of Eseutil.
Table 1 Modes of Eseutil operation
Mode of operation | Task |
---|---|
Eseutil /d |
Performs an offline compaction of a database. |
Eseutil /r |
Performs soft recovery to bring a single database into a consistent or clean shutdown state. |
Eseutil /g |
Verifies the integrity of a database. |
Eseutil /m |
Generates formatted output of various database file types. |
Eseutil /p |
Repairs a corrupted or damaged database. |
Eseutil /c |
Performs a hard recovery after a database restore. |
Eseutil /k |
Verifies the checksums of a database. |
Eseutil /y |
Copies a database, streaming file, or log file. |
For more information about defragmenting Exchange 2000 Server and Exchange Server 2003 databases, see Microsoft Knowledge Base article 192185, "How to Defragment with the Eseutil Utility (Eseutil.exe)," (https://go.microsoft.com/fwlink/?LinkId=3052&kbid=192185).
For more information about defragmenting Exchange Server databases, see Microsoft Knowledge Base article 328804, "How to defragment Exchange databases," (https://go.microsoft.com/fwlink/?LinkId=3052&kbid=328804).
You can also obtain information in the Microsoft Exchange Team Blog, "Is offline defragmentation considered regular Exchange maintenance?" (https://go.microsoft.com/fwlink/?linkid=35348).
Verify Mailbox and Public Folder Store Integrity
You will need to verify the Exchange store integrity in the following situations:
An item count on a mailbox or public folder store is inconsistent. This may indicate that some of the counters and pointers in your mailbox or public folder store may be corrupted.
You cannot move a mailbox. For example, if the Move Mailbox command or ExMerge fails on a particular mailbox, the structure of the mailbox, or of a message inside the mailbox, may be corrupted.
The e-mail client crashes frequently. For example, Microsoft Office Outlook® crashes repeatedly when a user tries to access a particular message or mailbox. Running a store integrity check may identify the cause of the errors.
Isinteg
Isinteg is a command-line tool that searches an offline Exchange store for integrity weaknesses. You can also repair issues that Isinteg detects. The Microsoft Exchange Information Store service must be online when Isinteg is started
By default, Isinteg is located in the <drive>:\<installation root>\exchsrvr\bin directory after running Exchange Server 2000 or Exchange Server 2003 Setup (where <drive> is the drive letter of the drive on which you installed Exchange Server).
The Isinteg tool performs the following tasks:
Verifies whether the MSExchangeIS service is stopped, and then Isinteg does one of the following tasks:
If the MSExchangeIS service is stopped, Isinteg displays the following error message: “Error: unable to get databases status from server." The reason could be either an incorrect server name or networking problems.
If the service is not stopped, Isinteg displays a list of databases to select from on that server.
Browses all the cross-reference tables for errors. Isinteg builds an Exchange database, Refer.mdb, of reference counts before it browses the tables.
Compares the counts found to the counts in the reference database. If Isinteg is running with the -fix switch, these counts are updated to the true values, as determined by Isinteg.
Performs the named to ID or named properties cleanup check to remove unused named properties.
Important
Isinteg cannot resolve physical database problems. If the database is damaged on the physical database page level (for example, because of hard disk problems, the file-level antivirus software modified the database, and so on), you may need to use the Eseutil tool. The database must be in the “clean shutdown” state for Isinteg to run on it.
Verifying the Exchange Store Integrity
To view the command-line Help about Isinteg tool, type the following command line at a command prompt:
<drive>:\<installation root>\exchsrvr\bin>isinteg /?
The syntax for using the Isinteg command-line tool is as follows:
isinteg -s ServerName [-fix] [-verbose] [-l LogFilename] -test TestName[[, TestName]...]
Use the information in Table 2 to determine which switch to use when you run the Isinteg tool.
Table 2 Isinteg switches
Switch | Use the switch to |
---|---|
-fix |
Fix any inconsistencies in your database. |
-verbose |
Display a detailed report of issues that Isinteg discovers. |
-test TestName |
Define the tests that Isinteg will perform when it runs (for example, to perform all tests available, use –test alltests, or to test folders, use –test allfoldertests). |
-dump |
Create a verbose dump file of store data. |
At a minimum, you must specify -test Testname or -dump at the command line to be able to proceed.
The steps to run the Isinteg command-line tool based on a specific criteria are as follows:
- To test the integrity of the Exchange store, at a command prompt, type:
<drive>:\<installation root>\exchsrvr\bin>isinteg -s ServerName -test alltests
For example, you can type c:\program files\exchsrvr\bin>isinteg -s servername -fix -test alltests
- To fix inconsistencies and errors in the Exchange store, at a command prompt, type:
<drive>:\<installation root>\exchsrvr\bin>isinteg -s ServerName -fix
For more information about the Isinteg tool, see Microsoft Knowledge Base article 182081, "Description of the Isinteg utility" (https://go.microsoft.com/fwlink/?LinkId=3052&kbid=182081).
For more information about the command-line parameters used for Isinteg tool, see Microsoft Knowledge Base article 301460, "Exchange Command-Line Parameters for the Isinteg.exe Tool" (https://go.microsoft.com/fwlink/?LinkId=3052&kbid=301460).
Checking the Queue Viewer
Queue Viewer is a tool that lets you maintain and administer your organization's messaging queues and identify mail-flow issues. Exchange uses queues to hold messages as they are being processed for routing and delivery. Queue Viewer is available on all SMTP virtual servers, X.400 objects, and all installed Microsoft Exchange Connectors for Novell GroupWise, Lotus Notes, and Lotus cc:Mail.
You must develop a queue baseline so that you can identify the difference between normal behavior and abnormal behavior for your organization. Typically, on-demand use of Queue Viewer is the result of a support call indicating that e-mail delivery is slow or a message has not been delivered.
For more information about Queue Viewer, see the topic "Checking Message Paths" in the topic "Daily Operations."
For more information about each queue, common causes of problems in each queue, and how you can troubleshoot mail-flow issues, see Microsoft Knowledge Base article 823489, "How to Use Queue Viewer to Troubleshoot Mail Flow Issues" (https://go.microsoft.com/fwlink/?LinkId=3052&kbid=823489).
You can use Queue Viewer to check for the following:
Messages that are queued for extended periods of time. Unless an Exchange 2003 server handles an extremely high volume of e-mail messages, the server will not typically have queued messages for any extended duration. Having extended periods of queuing typically indicates a system issue that warrants your attention. Review your performance metrics to see if some other performance issues are causing e-mail messages to queue. If not, look for connectors or servers that are down or not functioning. SMTP protocol logging might also help you discover the problem.
Peaks in queued messages. Spikes in queued messages can occur when someone sends:
A message to a large distribution list.
An extremely large message to many people.
A message whose destination is across a slow network link.
These conditions are not cause for alarm. However, you must review the security of your Exchange organization if either of the following conditions exists:
A large volume of messages is queued to one recipient or e-mail address. A large volume of messages queued to one recipient or e-mail address can be a symptom of a spam attack on an e-mail loop or a denial-of-service (DoS) attack.
A large volume of messages is queued to a specific server or domain. A large volume of messages queued to a particular server or domain indicates that a server is down, a service is stopped, a domain is unreachable, or a network disruption is preventing the system from establishing a connection.
Guidelines for Configuring a Performance Console
You can configure System Monitor to monitor the performance of your Exchange servers to understand what system behavior is normal and what changes you can make to improve Exchange performance.
You must be able to answer some basic questions about your system environment, including the following:
Number of messages received per user per day
Number of messages that are downloaded
Frequency with which users open folders
Number of additional users who servers can support
Peak delivery rate, the peak period during the day, and the peak day of the week
Frequency of monthly or quarterly peaks, if any
You must create a Performance console that lets you see the entire system environment and also register even minor changes in the performance of your servers. The guidelines for creating a Performance console are as follows:
Create a Performance console with two charts that have two different sample times, such as:
900 seconds for a 24-hour view
10 seconds to catch short-lived spikes
Include a minimal set of counters in each console, such as:
Processor(_Total)\% Processor Time
Process(store)\% Processor Time
MSExchangeIS\RPC Requests
MSExchangeIS\RPC Operations/sec
MSExchangeIS\RPC Averaged Latency
PhysicalDisk(_Total)\Disk Transfers/sec
PhysicalDisk(_Total)\Avg. Disk sec/Read
PhysicalDisk(_Total)\Avg. Disk sec/Write
SMTP Server\Local Queue Length
SMTP Server\Messages Delivered/sec
MSExchangeIS Mailbox\Local Delivery Rate
MSExchangeIS Mailbox\Folder Opens/sec
MSExchangeIS Mailbox\Message Opens/sec
Examine your busiest server to gather information about why the server is so busy and to understand what performance issues can be resolved when the server does not perform as well as the other servers.
Save reference log files in order to develop historical baseline data that will enable you to see what changes have occurred so you can then deal with them in growth over time.
For more information about creating a performance baseline for your server, see Troubleshooting Microsoft Exchange Server 2003 Performance (https://go.microsoft.com/fwlink/?LinkId=22811).