How we in Exchange support troubleshoot this issue is by first looking at the message ID.  If the message ID of the message is the same, then the root problem points to the Exchange Information Store service.  If the message ID is different for each duplicate message (usually incremented by 1) then the problem points to a client.  What’s behind this is that if a client is submitting a message more than once, the message will get a new message ID.  If the Exchange Store service see’s the message as not being delivered for some reason, it will deliver the same message (same ID) more than once.  Duplicate messages are usually the fault of a client submitting the message more than once.

Problem

Users intermittently receive duplicate, triplicate, etc. mail messages.

Cause

Two or more Outlook clients are logged into the same mailbox at the same time and at least one of the clients has a service installed in the profile that uses the “TransportSend()” function.

More Information

Message ID’s will different and usually be incremented by 1.  The “double submissions” (sometimes more) is due to race condition between two Outlook clients that are both connected in online mode to the same Exchange mailbox.  One of the clients more than likely have a ‘service’ enabled in the profile that uses a function called “TransportSend()”.  Some examples of this service are:

Live Meeting

Pop 3

Fax Services

Anti-Virus Agents (that typically send mail)

Any service that the type is ‘MAPI’

In this configuration, any sent message is placed into the spooler queue at submit time.  After, the spoolers from both clients ‘wake up’ and race with each other.  The spooler performs a fast-transfer copy of the original message into MDB Transport Queue folder and then sends ropTransportSend to submit the copy message.   At this moment, a ‘MailSubmitted’ event is generated for the copy message and it is processed by Exchange transport.  Original message is then moved to Sent Items by the client. (using non-spooler logon).

The double submit is possible if both spoolers work at about the same speed and both succeed in copying the submitted message into MDB Transport Queue folder.

The most common scenario that I observed while troubleshooting this issue is that a user would leave themselves logged on one computer (home/work) and then use another client computer.  One of the profiles on one of the client machines had another service in addition to the “Microsoft Exchange Server” service installed in the profile.

These examples show MAPI, however it is not limited to just MAPI:

Resolution

Only log into one client at a time or uninstall the additional service in the profiles.

The Exchange Troubleshooting Resolution Assistant or ExTRA is a powerful tracing tool that Exchange Escalation engineers use almost daily.  Tools like ExTRA are usually called trace “enablers” because they simply enable the ETW tracing for a given module.  ExTRA is Microsoft Exchange’s trace “enabler”.  Unfortunately, there have been reports of missing trace data when using ExTRA to obtain traces.  To avoid this data loss, you can use Performance Monitor to obtain the traces.  Performance Monitor also allows for additional configuration options.

Overview

To use this method, you still have to use ExTRA as the “enabler” to enable the tags you’d like to be traced.  After the desired tags are selected and enabled through ExTRA, you’ll then revert to Performance Monitor to obtain the actual trace. 

4.       Select the desired tracing to be enabled.  Be sure to select the “Info, Debug, Performance, and Function” Trace Types for maximum trace output.

5.       Click on Start Tracing.

6.       Immediately after starting the trace in ExTRA, stop the trace by clicking “Stop tracing now”.  The actual tracing will be done using Performance Monitor (PerfMon).  We are only using ExTRA to enable the desired tags.

7.       Now Start Performance Monitor. (Start, Administrative Tools, Performance) (Windows 2003)

8.       Navigate under Performance Logs and Alerts and find “Trace Logs”.

9.       Right click on Trace Logs and select “New Log Settings”.

10.   Give your new log a name like “MyTrace”.  Click OK.

11.   Under “Nonsystem providers” click Add.  Scroll down to “Microsoft Exchange Server 2007” and click OK. 

Note – There are other Non-system providers such as “Exchange Information Store”.  These other providers will not produce the desired results and should not be selected when doing ExTRA tracing.

12.   Navigate to the “Log Files” tab and click “Configure”.

13.   Here’s where we can configure settings such as the location and size limit of the file.  It is best to use a smaller size so that the trace files are easier to manage after conversion and it is also better for transportation.

14.   Navigate to the “Schedule” tab.  Select to start the log ‘manually’ if so desired but most important, select to stop the log “When the XX-MB log file is full”.  Also select, “Start a new log file”.

15.   Select OK.

16.   We have setup the log capture.  To start it, just right click and select Start.  The log can be stopped and started without overwriting previous logs.