Dmitry Khrebin's Exchange blog

Why Exchange 2013 CU6+ use out-of-site DCs/GCs

Khrebin Dmitry — Wed, 12 Aug 2015 03:33:00 GMT

This is no more applicable post Exchange 2013 CU11 for On-premise environments - also not valid for Exchange 2016! We have few escalations from our customers, who recognized huge traffic between Exchange 2013 CU6+ and out-of-site DCs/GCs. When we Get-ExchangeServer –Status we can see that Exchange uses out-of-site DCs but at a same time in event 2080 we can see that other In-Site DC are availible. Here is how it looks in Exchange 2010 and Exchange 2013 RTM – CU5: I used topology with 4 DC in-Site and 1 Out-Site From event 2080: Process Microsoft.Exchange.Directory.TopologyService.exe (PID=2276). Exchange Active Directory Provider has discovered the following servers with the following characteristics: (Server name | Roles | Enabled | Reachability | Synchronized | GC capable | PDC | SACL right | Critical Data | Netlogon | OS Version) In-site: DC001.CU1.com CDG 1 7 7 1 0 1 1 7 1 dc2.CU1.com CDG 1 7 7 1 0 1 1 7 1 DC3.CU1.com CDG 1 7 7 1 0 1 1 7 1 dc4.CU1.com CDG 1 7 7 1 0 1 1 7 1 Out-of-site: dc5.CU1.com CDG 1 7 7 1 0 1 1 7 1 Get-ExchangeServer exch5-cu1 -Status CurrentDomainControllers : {dc2.CU1.com, DC001.CU1.com, dc4.CU1.com, DC3.CU1.com} CurrentGlobalCatalogs : {dc2.CU1.com, DC001.CU1.com, dc4.CU1.com, DC3.CU1.com} CurrentConfigDomainController : DC001.CU1.com >netstat -n | findstr 3268 We can see established connections with all 4 GCs Turn off DC4 Information MSExchange ADAccess 2070 Topology: Process MSExchangeHMWorker.exe (ExHMWorker) (PID=3116). Exchange Active Directory Provider lost contact with domain controller dc4.CU1.com. Error was 0x34 (Unavailable) (Active directory response: The server is unavailable.). Exchange Active Directory Provider will attempt to reconnect with this domain controller when it is reachable. Get-ExchangeServer exch5-cu1 -Status CurrentDomainControllers : {DC001.CU1.com, DC3.CU1.com, dc2.CU1.com } CurrentGlobalCatalogs : {DC001.CU1.com, DC3.CU1.com, dc2.CU1.com} CurrentConfigDomainController : DC001.CU1.com >netstat -n | findstr 3268 We can see established connections with 3 GCs Turn off DC3 CurrentDomainControllers : {DC001.CU1.com, dc2.CU1.com} CurrentGlobalCatalogs : {DC001.CU1.com, dc2.CU1.com} >netstat -n | findstr 3268 We can see established connections with 2 In-Site GCs Turn off DC2 CurrentDomainControllers : {DC001.CU1.com} CurrentGlobalCatalogs : {DC001.CU1.com} CurrentConfigDomainController : DC001.CU1.com >netstat -n | findstr 3268 We can see connections only to DC001 From event 2080: Process Microsoft.Exchange.Directory.TopologyService.exe (PID=2276). Exchange Active Directory Provider has discovered the following servers with the following characteristics: (Server name | Roles | Enabled | Reachability | Synchronized | GC capable | PDC | SACL right | Critical Data | Netlogon | OS Version) In-site: DC001.CU1.com CDG 1 7 7 1 0 1 1 7 1 dc2.CU1.com CDG 1 0 0 0 0 0 0 0 0 DC3.CU1.com CDG 1 0 0 0 0 0 0 0 0 dc4.CU1.com CDG 1 0 0 0 0 0 0 0 0 Out-of-site: dc5.CU1.com CDG 1 7 7 1 0 1 1 7 1 In other words: we do not try to establish connection to Out-of-site DC while we have at least one In-site DC availible. What happenes as soon as you update your servers to CU6+: Get-ExchangeServer exch5-cu1 -Status CurrentDomainControllers : {dc2.CU1.com, DC001.CU1.com, dc4.CU1.com, DC3.CU1.com} CurrentGlobalCatalogs : {dc2.CU1.com, DC001.CU1.com, dc4.CU1.com, DC3.CU1.com} CurrentConfigDomainController : DC001.CU1.com >netstat -n | findstr 3268 We can see established connections with all 4 GCs Same as in RTM Turn off DC4 Get-ExchangeServer exch5-cu1 -Status CurrentDomainControllers : {DC001.CU1.com, DC3.CU1.com, dc2.CU1.com } CurrentGlobalCatalogs : {DC001.CU1.com, DC3.CU1.com, dc2.CU1.com} CurrentConfigDomainController : DC001.CU1.com >netstat -n | findstr 3268 We can see established connections with 3 GCs Same as RTM Turn off DC3 CurrentDomainControllers : {DC001.CU1.com, dc2.CU1.com, dc5.CU1.com} CurrentGlobalCatalogs : {DC001.CU1.com, dc2.CU1.com, dc5.CU1.com} CurrentConfigDomainController : DC001.CU1.com NEW!!! We established connection to Out-of-Site DC dc5.cu1.com It is by design. Saying that if number of in-site DCs are less than MinSuitableServer, which is by default 3, out-site DCs will be used. Once the number of in-site DCs is larger than MinSuitableServer, out-site DCs should not be used any more. Previously when Exchange process asks for domain controllers, topology service only returns servers from either In-Site list or Out-of-Site list. That says, as long as there is one single DC suitable in In-Site list, topology service will return it back and does not further search Out-of-Site list, no matter how many is requested by the client.

This might cause some load unbalanced issue, especially during site failover. Good domain controllers left in the being failed out site take much more load than outside DCs.

To fix this, a new configurable setting, MinSuitableServer, is introduced. Topology service will first check whether there are enough suitable servers in In-Site list. If no, it will add servers from Out-of-Site list. Similar change is done in topology discovery, too. How we can return it back or configure? If we really want to use in-site DCs only, even though there is just 1 available (as it was in 2010 or 2013 RTM-CU5), we can add an entry: MinSuitableServer = "1" in Microsoft.Exchange.Directory.TopologyService.exe.config: In section I turned DC4 off as we do not need it Also I added MinSuitableServer = "2" and restarted Microsoft Exchange Active Directory Topology aka MSExchangeADTopology or whole server CurrentDomainControllers : {DC3.CU1.com, dc2.CU1.com, DC001.CU1.com} CurrentGlobalCatalogs : {DC3.CU1.com, dc2.CU1.com, DC001.CU1.com} CurrentConfigDomainController : dc2.CU1.com Turn DC3 off From event 2080: Process Microsoft.Exchange.Directory.TopologyService.exe (PID=2504). Exchange Active Directory Provider has discovered the following servers with the following characteristics: (Server name | Roles | Enabled | Reachability | Synchronized | GC capable | PDC | SACL right | Critical Data | Netlogon | OS Version) In-site: DC001.CU1.com CDG 1 7 7 1 0 1 1 7 1 dc2.CU1.com CDG 1 7 7 1 0 1 1 7 1 DC3.CU1.com CDG 1 0 0 0 0 0 0 0 0 dc4.CU1.com CDG 1 0 0 0 0 0 0 0 0 Out-of-site: dc5.CU1.com CDG 1 7 7 1 0 1 1 7 1 [PS] C:\Windows\system32>Get-ExchangeServer Exch5-cu1 -Status | fl Current* CurrentDomainControllers : {DC001.CU1.com, dc2.CU1.com} CurrentGlobalCatalogs : {DC001.CU1.com, dc2.CU1.com} CurrentConfigDomainController : dc2.CU1.com Turn off DC2 CurrentDomainControllers : {DC001.CU1.com, dc5.CU1.com} CurrentGlobalCatalogs : {DC001.CU1.com, dc5.CU1.com} CurrentConfigDomainController : DC001.CU1.com Start DC3 CurrentDomainControllers : {DC001.CU1.com, DC3.CU1.com} CurrentGlobalCatalogs : {DC001.CU1.com, DC3.CU1.com} CurrentConfigDomainController : DC001.CU1.com So we returned back to In-site DC as soon it became available. Now set MinSuitableServer = "1" CurrentDomainControllers : {dc2.CU1.com, DC3.CU1.com, DC001.CU1.com} CurrentGlobalCatalogs : {dc2.CU1.com, DC3.CU1.com, DC001.CU1.com} CurrentConfigDomainController : DC001.CU1.com Turn off DC2 CurrentDomainControllers : {DC3.CU1.com, DC001.CU1.com} CurrentGlobalCatalogs : {DC3.CU1.com, DC001.CU1.com} CurrentConfigDomainController : DC001.CU1.com Turn off DC3 CurrentDomainControllers : {DC001.CU1.com} CurrentGlobalCatalogs : {DC001.CU1.com} CurrentConfigDomainController : DC001.CU1.com In other words: same as it were in 2010 and 2013 RTM-CU5.

How rebase appointments using Microsoft Office Outlook Tool: Time Zone Data Update Tool for Microsoft Office Outlook.

Khrebin Dmitry — Wed, 01 Oct 2014 09:00:00 GMT

I used appointments and envirement mentioned in Update for Russian Time Zone Changes 2014 and Exchange (How auto-adjustment works)

As some of our colleagues still use legacy clients we can try to resolve this issue by using Microsoft Office Outlook Tool: Time Zone Data Update Tool for Microsoft Office Outlook 2010 64-bit I didn't find version for Outlook 2013 (for Outlook 2010 32bit and legacy versions and possiblu for 2013 32x)

First attempt: Try to move user to the new TZ:

Tool found 6 appointments

Here is log with details:

[Original Time Zone]

(UTC+03:00) Moscow, St. Petersburg, Volgograd (RTZ 2)

[New Time Zone]

(UTC+03:00) Moscow, St. Petersburg, Volgograd (RTZ 2) (Update)

[Time Zone Update Log]

Type: Meeting

ID: 040000008200e00074c5b7101a82e00800000000f03d1461bbd0cf01000000000000000010000000aa51f5623a26174d859f60e320ad65fe

Subject: User2010@OWA2010 15:00

Old Start Time: Tuesday, December 2, 2014 11:00:00 AM

New Start Time: Tuesday, December 2, 2014 12:00:00 PM

Old End Time: Tuesday, December 2, 2014 12:00:00 PM

New End Time: Tuesday, December 2, 2014 1:00:00 PM

Recurring: Yes

Result: Success

Type: Meeting

ID: 040000008200e00074c5b7101a82e00800000000504a16dcdad0cf0100000000000000001000000057b25eceaa079d4d92eacebf16bd8f49

Subject: User2010@Outlook2010 12:00

Old Start Time: Tuesday, December 2, 2014 8:00:00 AM

New Start Time: Tuesday, December 2, 2014 9:00:00 AM

Old End Time: Tuesday, December 2, 2014 9:00:00 AM

New End Time: Tuesday, December 2, 2014 10:00:00 AM

Recurring: Yes

Result: Success

Type: Meeting

ID: 040000008200e00074c5b7101a82e00800000000ce35713bbbd0cf01000000000000000010000000095ddfdd19ccdb40bc0fa284fe2025f3

Subject: User2010@OWA2010 15:00

Old Start Time: Tuesday, November 4, 2014 11:00:00 AM

New Start Time: Tuesday, November 4, 2014 12:00:00 PM

Old End Time: Tuesday, November 4, 2014 12:00:00 PM

New End Time: Tuesday, November 4, 2014 1:00:00 PM

Recurring: No

Result: Success

Type: Meeting

ID: 040000008200e00074c5b7101a82e00800000000e086abb2d9d0cf0100000000000000001000000045f49c81cbae284e838ee937d3e6bddd

Subject: User2010@Outlook2010 12:00

Old Start Time: Tuesday, November 4, 2014 8:00:00 AM

New Start Time: Tuesday, November 4, 2014 9:00:00 AM

Old End Time: Tuesday, November 4, 2014 9:00:00 AM

New End Time: Tuesday, November 4, 2014 10:00:00 AM

Recurring: No

Result: Success

Type: Meeting

ID: 040000008200e00074c5b7101a82e008000000000dab4c7ebbd0cf010000000000000000100000000dfcacc47e1592478ebc26e8f31d36f8

Subject: User2010@OWA2010 15:00

Old Start Time: Tuesday, October 14, 2014 11:00:00 AM

New Start Time: Tuesday, October 14, 2014 11:00:00 AM

Old End Time: Tuesday, October 14, 2014 12:00:00 PM

New End Time: Tuesday, October 14, 2014 12:00:00 PM

Recurring: Yes

Result: Success

Type: Meeting

ID: 040000008200e00074c5b7101a82e0080000000080c7ffaedad0cf010000000000000000100000007af4d49f3ef2944887245843eccafd92

Subject: User2010@Outlook2010 12:00

Old Start Time: Tuesday, October 14, 2014 8:00:00 AM

New Start Time: Tuesday, October 14, 2014 8:00:00 AM

Old End Time: Tuesday, October 14, 2014 9:00:00 AM

New End Time: Tuesday, October 14, 2014 9:00:00 AM

Recurring: Yes

Result: Success

Okay, also we sent batch updates to other users:

In User2010’s calendar, everything is fine (BTW if we do same with Outlook 2010 for User2013 - same results in Outlook 2010 for User2010):

Single appointments

Recurrent appointments:

What did we do with appointment in organizer's mailbox?

Lets have a look

We changed PidLidAppointmentStartWhole and PidLidAppointmentEndWhole:

Also we updated PidLidAppointmentTimeZoneDefinitionStartDisplay and PidLidAppointmentTimeZoneDefinitionEnsDisplay (now it is insanely long)

Time Zone Definition:

bMajorVersion = 0x02 (2)

bMinorVersion = 0x01 (1)

cbHeader = 0x0030 (48)

wReserved = 0x0002 (2)

cchKeyName = 0x0015 (21)

szKeyName = Russian Standard Time

cRules = 0x0006 (6)

TZRule[0x0].bMajorVersion = 0x02 (2)

TZRule[0x0].bMinorVersion = 0x01 (1)

TZRule[0x0].wReserved = 0x003E (62)

TZRule[0x0].wTZRuleFlags = 0x0000 = 0x0

TZRule[0x0].wYear = 0x07DA (2010)

TZRule[0x0].X = cb: 14 lpb: 0100000001000000000000000000

TZRule[0x0].lBias = 0xFFFFFF4C (-180)

TZRule[0x0].lStandardBias = 0x00000000 (0)

TZRule[0x0].lDaylightBias = 0xFFFFFFC4 (-60)

TZRule[0x0].stStandardDate.wYear = 0x0 (0)

TZRule[0x0].stStandardDate.wMonth = 0xA (10)

TZRule[0x0].stStandardDate.wDayOfWeek = 0x0 (0)

TZRule[0x0].stStandardDate.wDay = 0x5 (5)

TZRule[0x0].stStandardDate.wHour = 0x3 (3)

TZRule[0x0].stStandardDate.wMinute = 0x0 (0)

TZRule[0x0].stStandardDate.wSecond = 0x0 (0)

TZRule[0x0].stStandardDate.wMilliseconds = 0x0 (0)

TZRule[0x0].stDaylightDate.wYear = 0x0 (0)

TZRule[0x0].stDaylightDate.wMonth = 0x3 (3)

TZRule[0x0].stDaylightDate.wDayOfWeek = 0x0 (0)

TZRule[0x0].stDaylightDate.wDay = 0x5 (5)

TZRule[0x0].stDaylightDate.wHour = 0x2 (2)

TZRule[0x0].stDaylightDate.wMinute = 0x0 (0)

TZRule[0x0].stDaylightDate.wSecond = 0x0 (0)

TZRule[0x0].stDaylightDate.wMilliseconds = 0x0 (0)

TZRule[0x1].bMajorVersion = 0x02 (2)

TZRule[0x1].bMinorVersion = 0x01 (1)

TZRule[0x1].wReserved = 0x003E (62)

TZRule[0x1].wTZRuleFlags = 0x0000 = 0x0

TZRule[0x1].wYear = 0x07DB (2011)

TZRule[0x1].X = cb: 14 lpb: 0100000001000000000000000000

TZRule[0x1].lBias = 0xFFFFFF4C (-180)

TZRule[0x1].lStandardBias = 0x00000000 (0)

TZRule[0x1].lDaylightBias = 0xFFFFFFC4 (-60)

TZRule[0x1].stStandardDate.wYear = 0x0 (0)

TZRule[0x1].stStandardDate.wMonth = 0x1 (1)

TZRule[0x1].stStandardDate.wDayOfWeek = 0x6 (6)

TZRule[0x1].stStandardDate.wDay = 0x1 (1)

TZRule[0x1].stStandardDate.wHour = 0x0 (0)

TZRule[0x1].stStandardDate.wMinute = 0x0 (0)

TZRule[0x1].stStandardDate.wSecond = 0x0 (0)

TZRule[0x1].stStandardDate.wMilliseconds = 0x0 (0)

TZRule[0x1].stDaylightDate.wYear = 0x0 (0)

TZRule[0x1].stDaylightDate.wMonth = 0x3 (3)

TZRule[0x1].stDaylightDate.wDayOfWeek = 0x0 (0)

TZRule[0x1].stDaylightDate.wDay = 0x5 (5)

TZRule[0x1].stDaylightDate.wHour = 0x2 (2)

TZRule[0x1].stDaylightDate.wMinute = 0x0 (0)

TZRule[0x1].stDaylightDate.wSecond = 0x0 (0)

TZRule[0x1].stDaylightDate.wMilliseconds = 0x0 (0)

TZRule[0x2].bMajorVersion = 0x02 (2)

TZRule[0x2].bMinorVersion = 0x01 (1)

TZRule[0x2].wReserved = 0x003E (62)

TZRule[0x2].wTZRuleFlags = 0x0000 = 0x0

TZRule[0x2].wYear = 0x07DC (2012)

TZRule[0x2].X = cb: 14 lpb: 0100000001000000000000000000

TZRule[0x2].lBias = 0xFFFFFF10 (-240)

TZRule[0x2].lStandardBias = 0x00000000 (0)

TZRule[0x2].lDaylightBias = 0xFFFFFFC4 (-60)

TZRule[0x2].stStandardDate.wYear = 0x0 (0)

TZRule[0x2].stStandardDate.wMonth = 0x0 (0)

TZRule[0x2].stStandardDate.wDayOfWeek = 0x0 (0)

TZRule[0x2].stStandardDate.wDay = 0x0 (0)

TZRule[0x2].stStandardDate.wHour = 0x0 (0)

TZRule[0x2].stStandardDate.wMinute = 0x0 (0)

TZRule[0x2].stStandardDate.wSecond = 0x0 (0)

TZRule[0x2].stStandardDate.wMilliseconds = 0x0 (0)

TZRule[0x2].stDaylightDate.wYear = 0x0 (0)

TZRule[0x2].stDaylightDate.wMonth = 0x0 (0)

TZRule[0x2].stDaylightDate.wDayOfWeek = 0x0 (0)

TZRule[0x2].stDaylightDate.wDay = 0x0 (0)

TZRule[0x2].stDaylightDate.wHour = 0x0 (0)

TZRule[0x2].stDaylightDate.wMinute = 0x0 (0)

TZRule[0x2].stDaylightDate.wSecond = 0x0 (0)

TZRule[0x2].stDaylightDate.wMilliseconds = 0x0 (0)

TZRule[0x3].bMajorVersion = 0x02 (2)

TZRule[0x3].bMinorVersion = 0x01 (1)

TZRule[0x3].wReserved = 0x003E (62)

TZRule[0x3].wTZRuleFlags = 0x0000 = 0x0

TZRule[0x3].wYear = 0x07DD (2013)

TZRule[0x3].X = cb: 14 lpb: 0100000001000000000000000000

TZRule[0x3].lBias = 0xFFFFFF10 (-240)

TZRule[0x3].lStandardBias = 0x00000000 (0)

TZRule[0x3].lDaylightBias = 0xFFFFFFC4 (-60)

TZRule[0x3].stStandardDate.wYear = 0x0 (0)

TZRule[0x3].stStandardDate.wMonth = 0x0 (0)

TZRule[0x3].stStandardDate.wDayOfWeek = 0x0 (0)

TZRule[0x3].stStandardDate.wDay = 0x0 (0)

TZRule[0x3].stStandardDate.wHour = 0x0 (0)

TZRule[0x3].stStandardDate.wMinute = 0x0 (0)

TZRule[0x3].stStandardDate.wSecond = 0x0 (0)

TZRule[0x3].stStandardDate.wMilliseconds = 0x0 (0)

TZRule[0x3].stDaylightDate.wYear = 0x0 (0)

TZRule[0x3].stDaylightDate.wMonth = 0x0 (0)

TZRule[0x3].stDaylightDate.wDayOfWeek = 0x0 (0)

TZRule[0x3].stDaylightDate.wDay = 0x0 (0)

TZRule[0x3].stDaylightDate.wHour = 0x0 (0)

TZRule[0x3].stDaylightDate.wMinute = 0x0 (0)

TZRule[0x3].stDaylightDate.wSecond = 0x0 (0)

TZRule[0x3].stDaylightDate.wMilliseconds = 0x0 (0)

TZRule[0x4].bMajorVersion = 0x02 (2)

TZRule[0x4].bMinorVersion = 0x01 (1)

TZRule[0x4].wReserved = 0x003E (62)

TZRule[0x4].wTZRuleFlags = 0x0002 = TZRULE_FLAG_EFFECTIVE_TZREG

TZRule[0x4].wYear = 0x07DE (2014)

TZRule[0x4].X = cb: 14 lpb: 0100000001000000000000000000

TZRule[0x4].lBias = 0xFFFFFF4C (-180)

TZRule[0x4].lStandardBias = 0x00000000 (0)

TZRule[0x4].lDaylightBias = 0xFFFFFFC4 (-60)

TZRule[0x4].stStandardDate.wYear = 0x0 (0)

TZRule[0x4].stStandardDate.wMonth = 0xA (10)

TZRule[0x4].stStandardDate.wDayOfWeek = 0x0 (0)

TZRule[0x4].stStandardDate.wDay = 0x5 (5)

TZRule[0x4].stStandardDate.wHour = 0x2 (2)

TZRule[0x4].stStandardDate.wMinute = 0x0 (0)

TZRule[0x4].stStandardDate.wSecond = 0x0 (0)

TZRule[0x4].stStandardDate.wMilliseconds = 0x0 (0)

TZRule[0x4].stDaylightDate.wYear = 0x0 (0)

TZRule[0x4].stDaylightDate.wMonth = 0x1 (1)

TZRule[0x4].stDaylightDate.wDayOfWeek = 0x3 (3)

TZRule[0x4].stDaylightDate.wDay = 0x1 (1)

TZRule[0x4].stDaylightDate.wHour = 0x0 (0)

TZRule[0x4].stDaylightDate.wMinute = 0x0 (0)

TZRule[0x4].stDaylightDate.wSecond = 0x0 (0)

TZRule[0x4].stDaylightDate.wMilliseconds = 0x0 (0)

TZRule[0x5].bMajorVersion = 0x02 (2)

TZRule[0x5].bMinorVersion = 0x01 (1)

TZRule[0x5].wReserved = 0x003E (62)

TZRule[0x5].wTZRuleFlags = 0x0000 = 0x0

TZRule[0x5].wYear = 0x07DF (2015)

TZRule[0x5].X = cb: 14 lpb: 0100000001000000000000000000

TZRule[0x5].lBias = 0xFFFFFF4C (-180)

TZRule[0x5].lStandardBias = 0x00000000 (0)

TZRule[0x5].lDaylightBias = 0xFFFFFFC4 (-60)

TZRule[0x5].stStandardDate.wYear = 0x0 (0)

TZRule[0x5].stStandardDate.wMonth = 0x0 (0)

TZRule[0x5].stStandardDate.wDayOfWeek = 0x0 (0)

TZRule[0x5].stStandardDate.wDay = 0x0 (0)

TZRule[0x5].stStandardDate.wHour = 0x0 (0)

TZRule[0x5].stStandardDate.wMinute = 0x0 (0)

TZRule[0x5].stStandardDate.wSecond = 0x0 (0)

TZRule[0x5].stStandardDate.wMilliseconds = 0x0 (0)

TZRule[0x5].stDaylightDate.wYear = 0x0 (0)

TZRule[0x5].stDaylightDate.wMonth = 0x0 (0)

TZRule[0x5].stDaylightDate.wDayOfWeek = 0x0 (0)

TZRule[0x5].stDaylightDate.wDay = 0x0 (0)

TZRule[0x5].stDaylightDate.wHour = 0x0 (0)

TZRule[0x5].stDaylightDate.wMinute = 0x0 (0)

TZRule[0x5].stDaylightDate.wSecond = 0x0 (0)

TZRule[0x5].stDaylightDate.wMilliseconds = 0x0 (0)

In other words: for Organizer we rewrote all necessary attributes like if we recreated meeting with correct configuration. Cool!

On attendee's side:

We received a message with meeting update information:

However, single appointment updates do not contain TZ definition.

For recurrent appointments, we have TZ definition in update.

Also pay attention that it is 448 bytes long.

From user2007's client perspective everything is fine

Single appointments

Recurrent appointments

Cool!

Okay, next patient – User2013

Same updates (no surprise). However, a bit different user's experience.

Single appointments shifted (as we changed appointment start/end time in UTC, but didn't change TZ definition).

Recurrent appointment is fine (as we updated start end time and TZ definition).

Let’s try Daylight Saving Update Mode

Same update and same issue for single appointments

Okay, our problem is that if we update single appointments we do not update TZ definition. How can we manage it? Let’s check what happens if we try move from one zone to absolutely another one.

I'll move my User2010 to Atlantis (for whom else do we have -02:00 UTC TZ?)

Unfortunately, again we did not send TZ definition

Instead of recurrent meeting

And of cause no update for PidLidAppointmentTimeZoneDefinitionStartDisplay and PidLidAppointmentTimeZoneDefinitionEnsDisplay.

Time to return home

Issue is still here:

Conclusion:

Time Zone Data Update Tool for Microsoft Office Outlook works fine if all clients are legacy (2007). For clients 2010 and 2013 we see issue with single appointments (they «corrected» twice). In addition, it works fine for recurrent appointments. Unfortunately it not so careful for single appointments. On the other hand, single appointments will end eventually (instead of endless recurrent appointments).

Update for Russian Time Zone Changes 2014 and Exchange (How auto-adjustment works)

Khrebin Dmitry — Tue, 16 Sep 2014 00:49:00 GMT

Update from 17 of October 2014: Scripts to log and correct wrong or absent Time Zone deffinitions added.

As already, that is mentioned in https://blogs.technet.com/b/dst2007/archive/2014/08/22/announcement-update-for-russian-time-zone-changes.aspx Russia will change its existing time zones on October 26, 2014.

What Exchange may offer here to reduce pain for end users (as there are appointments shifted in time)?

For our case, we are interested in PidLidAppointmentTimeZoneDefinitionStartDisplay and PidLidAppointmentTimeZoneDefinitionEndDisplay (attribute description can be found here:
https://msdn.microsoft.com/en-us/library/ee160657(v=exchg.80).aspx)

Here it is:

What do we have in Smart View?

Time Zone Definition:

bMajorVersion = 0x02 (2)

bMinorVersion = 0x01 (1)

cbHeader = 0x0030 (48)

wReserved = 0x0002 (2)

cchKeyName = 0x0015 (21)

szKeyName = Russian Standard Time <- We use it to link appointment time zone definition with local registry definition

cRules = 0x0001 (1)

TZRule[0x0].bMajorVersion = 0x02 (2)

TZRule[0x0].bMinorVersion = 0x01 (1)

TZRule[0x0].wReserved = 0x003E (62)

TZRule[0x0].wTZRuleFlags = 0x0002 = TZRULE_FLAG_EFFECTIVE_TZREG

TZRule[0x0].wYear = 0x07DC (2012)

TZRule[0x0].X = cb: 14 lpb: 0100000001000000000000000000

TZRule[0x0].lBias = 0xFFFFFF10 (-240)

TZRule[0x0].lStandardBias = 0x00000000 (0)

TZRule[0x0].lDaylightBias = 0xFFFFFFC4 (-60)

TZRule[0x0].stStandardDate.wYear = 0x0 (0)

TZRule[0x0].stStandardDate.wMonth = 0x0 (0)

TZRule[0x0].stStandardDate.wDayOfWeek = 0x0 (0)

TZRule[0x0].stStandardDate.wDay = 0x0 (0)

TZRule[0x0].stStandardDate.wHour = 0x0 (0)

TZRule[0x0].stStandardDate.wMinute = 0x0 (0)

TZRule[0x0].stStandardDate.wSecond = 0x0 (0)

TZRule[0x0].stStandardDate.wMilliseconds = 0x0 (0)

TZRule[0x0].stDaylightDate.wYear = 0x0 (0)

TZRule[0x0].stDaylightDate.wMonth = 0x0 (0)

TZRule[0x0].stDaylightDate.wDayOfWeek = 0x0 (0)

TZRule[0x0].stDaylightDate.wDay = 0x0 (0)

TZRule[0x0].stDaylightDate.wHour = 0x0 (0)

TZRule[0x0].stDaylightDate.wMinute = 0x0 (0)

TZRule[0x0].stDaylightDate.wSecond = 0x0 (0)

TZRule[0x0].stDaylightDate.wMilliseconds = 0x0 (0)

Now let’s explain how it works (it will also explain why I did not use other combinations of Outlooks and Exchanges).

Clients (Outlook; OWA, EAS application pools) 2010+ when meeting is shown (sent to mobile device), compare TimeZone description (PidLidAppointmentTimeZoneDefinitionStartDisplay and PidLidAppointmentTimeZoneDefinitionEndDisplay ) in meeting with TZ description for the same Time Zone in local registry (2007-2010 CAS, 2013 MBX) on server for OWA, EAS protocols and on workstation for Outlook. Then if “client” understands, that TZ descriptions are different, it trusts local TZ definition (as it should be recent) and correct Named Property: id: 0x820D=33293 = PidLidAppointmentStartWhole, dispidApptStartWhole and Named Property: id: 0x820E=33294 = PidLidAppointmentEndWhole to shift appointment on the right time. Some details about how we store appointments here - https://blogs.technet.com/b/dkhrebin/archive/2012/12/07/ios-quot-quot-aka-exchange.aspx

In other words, we do not change anything in store. Appointments are stored untouched. Only clients correct them “on the fly”.

Now some test results.

Environment (all servers and clients updated with all recent updates from Windows Update).

Servers:

DC001 – 2012 Windows Server

Exch1 – 2007 SP3 RU14 MultiRole @ 2008 R2 SP1

Exch2 – 2010 SP3 RU7 MultiRole @ 2008 R2 SP1

Exch3 – 2013 CU6 CAFE @ 2012

Exch4 & Exch5– 2013 CU6 MBX @ 2012

Clients:

Client-Win7 – Windows 7 SP1 with Outlook 2007 (12.0.6691.5000) SP3 MSO (12.0.6683.5000)

Client-Win8-1 – Windows 8.1 with Outlook 2010 SP2 (14.0.7015.1000)

Client – Windows 8 with Outlook 2013 (15.0.4569.1503) MSO (15.0.4569.1506)

The following meetings were created:

Next, every other week meetings are created:

Next, every week meetings are created:

Now it is time to update our servers and workstations with new TZ definition.

I used this one for Moscow (hope we will have it in final release):

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Time Zones\Russian Standard Time]

"Display"="(UTC+03:00) Moscow, St. Petersburg, Volgograd (RTZ 2)"

"Dlt"="Russia TZ 2 Daylight Time"

"Std"="Russia TZ 2 Standard Time"

"MUI_Display"="@tzres.dll,-1830"

"MUI_Dlt"="@tzres.dll,-1831"

"MUI_Std"="@tzres.dll,-1832"

"TZI"=hex:4c,ff,ff,ff,00,00,00,00,c4,ff,ff,ff,00,00,0a,00,00,00,05,00,02,00,00,\

00,00,00,00,00,00,00,01,00,03,00,01,00,00,00,00,00,00,00,00,00

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Time Zones\Russian Standard Time\Dynamic DST]

"FirstEntry"=dword:000007da

"LastEntry"=dword:000007df

"2010"=hex:4c,ff,ff,ff,00,00,00,00,c4,ff,ff,ff,00,00,0a,00,00,00,05,00,03,00,\

00,00,00,00,00,00,00,00,03,00,00,00,05,00,02,00,00,00,00,00,00,00

"2011"=hex:4c,ff,ff,ff,00,00,00,00,c4,ff,ff,ff,00,00,01,00,06,00,01,00,00,00,\

00,00,00,00,00,00,00,00,03,00,00,00,05,00,02,00,00,00,00,00,00,00

"2012"=hex:10,ff,ff,ff,00,00,00,00,c4,ff,ff,ff,00,00,00,00,00,00,00,00,00,00,\

00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00

"2013"=hex:10,ff,ff,ff,00,00,00,00,c4,ff,ff,ff,00,00,00,00,00,00,00,00,00,00,\

00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00

"2014"=hex:4c,ff,ff,ff,00,00,00,00,c4,ff,ff,ff,00,00,0a,00,00,00,05,00,02,00,\

00,00,00,00,00,00,00,00,01,00,03,00,01,00,00,00,00,00,00,00,00,00

"2015"=hex:4c,ff,ff,ff,00,00,00,00,c4,ff,ff,ff,00,00,00,00,00,00,00,00,00,00,\

00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00

All clients (Outlooks 2007-2013 and OWA 2007-2013) show first single appointment at right time (not surprise as nothing changed)

Single appointments in Outlook 2007 after TZ changes:

Same for OWA 2007

Recurrent appointments also shifted.

Therefore, Outlook 2007 do not have code to correct appointments if TZ definition changed. Exchange 2007 need to be updated with SP3 RU15 to handle this issue KB3004235.

If you do not have 2010 or 2013 Exchange servers, you can follow https://blogs.technet.com/b/exchange/archive/2008/08/15/3406094.aspx

Outlook 2010, 2013 corrected meetings time for single and recurrence meetings

Same for OWA 2010 & 2013

Therefore, Exchange 2010+ and Outlook 2010+ can imidiatly correct appointments if TZ definition is changed.

Only fly in the ointment:

We need DST update for all Exchange servers to show correct Display names for TZs (expected in next RU/CU for all supported servers). Be aware: October 26 2014 Russian time zone changes and Exchange

Good news is that new TZs appears in ECP list immediately after server restart and are available to users:

In addition, pay attention, that Moscow is between Nairobi +03:00 and Tehran (+03:30), so it is sorted in a right way. You can use it to check if TZ was updated or not ;)

Thus if you use Outlook 2010 and legacy Exchange Outlook will shift appointment, but Exchange OWA and EAS not and vice versa.

Okay, if your users use only Outlook & OWA 2010+ (or already updated Exchange 2007 with SP3RU15) and none of them are in new TZs - lucky you.

Update your servers and users’ workstations with TZ patch.

In all other cases, we need some manual work: How rebase appointments using Microsoft Office Outlook Tool: Time Zone Data Update Tool for Microsoft Office Outlook.

Here we need to use Time Zone Data Update Tool for Microsoft Office https://www.microsoft.com/en-us/download/details.aspx?id=17291 or https://www.microsoft.com/en-us/download/details.aspx?id=16271 for x64 clients. Pay attention – for all appointments in your organization.

Other issue I found is bad TZ's stamps done by mobile devices!

If your users use EAS you can find the following issue for meetings created at 10:00 5/11 from iPhone and 12:00 4/11 from Nokia 920:

My 2013 clients Outlook and OWA cannot correct them.

Nokia 920 did not fill TZ description (PidLidAppointmentTimeZoneDefinitionStartDisplay and PidLidAppointmentTimeZoneDefinitionEndDisplay ) at all. Just wrote UTC for PidLidTimeZoneDescription https://msdn.microsoft.com/en-us/library/office/cc815321(v=office.15).aspx

Nokia 1520 wrote everything fine except szKeyName:

Hello! Who are you?

szKeyName = ?????????? ????? (????)

In hex: 003F003F003F003F003F003F003F003F003F003F0020003F003F003F003F003F00200028003F003F003F003F0029

Oh, yes. How we suppose to process it?

iPhone filled it (not sure why Europe/Moscow is (UTC) Monrovia, Reykjavik in PidLidTimeZoneDescription), may be it auto-filled if empty.

Here is TZ name

szKeyName = Europe/Moscow

Instead of

szKeyName = Russian Standard Time

Of cause, Apple better knows TZs in attributes hidden from users and used only by Exchange and Outlook.

Big problem, as Exchange and Outlook cannot link Europe/Moscow to Russian Standard Time and cannot correct these shifted meetings.

How can we manage it? Exsample of scripts (personal and server versions) which can find and if needed rewrite bad or missed attributes are on the bottom.

Please pay attention it is not offically supported solution from Microsoft!!!

It is just exsample how this kind of scripts can be written.

First of all I strongly reccomend use Test users/pilot groups without any corrections. Just logging what is issue scale, if there is.

Don't foget backup mailboxes if you decide to make corrections. Also expect metting update message storm!!!

Special thanks to Anna Chudova for co-authoring.

BadAppointmentsWithCorrection.zip

How Exchange IMAP Search in message header

Khrebin Dmitry — Fri, 04 Oct 2013 01:23:00 GMT

In case you or your script needs to search in message headers during IMAP session with
Exchange, please keep in mind next nuances:

To SEARCH for something in message header we can use SEARCH or SEARCH HEADER

If you use SEARCH FROM (for CC, BCC, TO idea the same) Exchange will search in message's Property: PR_SENT_REPRESENTING_NAME, PR_SENT_REPRESENTING_NAME_A,
ptagSentRepresentingName

The PR_SENT_REPRESENTING_NAME property contains the display name for the messaging user represented by the sender.

As fact, it is only name, not e-mail address

So if you receive message from me, you will see in P2 header:

From: Dmitry Khrebin dkhrebin@microsoft.com

Result is that ptagSentRepresentingName = Dmitry Khrebin

SMTP address stored in the separate Property: PidTagSenderSmtpAddress, ptagSenderSmtpAddress

And as a result you cannot find my message if try to search by e-mail address or by alias.

Only exception here – if no representing name in P2 header at all, like:

From: dkhrebin@microsoft.com

In this case, Exchange will copy e-mail address to ptagSentRepresentingName

The outgoing transport provider must always leave PR_SENT_REPRE represented sender properties unset.SENTING_NAME unchanged if it has been set by the sending client. If it is unset, the transport provider should set it to PR_SENDER_NAME on the outbound copy of the message, and leave it unset on the local copy.

Okey, what shall we do if we works with general messages with sender name in it?

Just use SEARCH HEADER

How it works:

SEARCH HEADER searches in Property: PR_TRANSPORT_MESSAGE_HEADERS, PR_TRANSPORT_MESSAGE_HEADERS_A

It is exactly what you can see in Outlook when open Message properties and look for Internet Headers

If we ask IMAP to SEARCH HEADER it will looks for

"\n" + fieldName + ":" + "*string*"

("\n" just to be sure that we look for beginning of the string to avoid false positive reaction in case somebody wrote in subject From: user@domain.com)

2003 Exchange ActiveSync's FE - BE interaction or how determine message from -FlatUrlSpace- URL

Khrebin Dmitry — Tue, 29 Jan 2013 23:59:00 GMT

Sometimes we need to determine what object exactly requested by client. This time I will show, how it can be done for activesync devices with Exchange 2003.

If we look in IIS logs on FrontEnd server, activesync's device request seems like this:
POST /Microsoft-Server-ActiveSync User=test-user&DeviceId=ApplDMPHTC56DVGJ&DeviceType=iPad&Cmd=Ping&Log=V4TNASNC:0A0C0D0FS:0A0C0D0SP:1C19I8487S49266R0S0L900H0P

On BackEnd server it will looks like:
/exchange/test-user@contoso.com/-FlatUrlSpace-/b284ef6a3c5c3c4b81caadcce6c72ec4-17A0/3b4222a57de41147b2e8cb424c0d9025-502eb

How we can interpreter it? Some description can be found at https://msdn.microsoft.com/en-us/library/ee237541(v=exchg.80).aspx

For folders: https://server/virtual_root/-FlatUrlSpace-/something-not-important-in-this-case-fid
For messages:

There must be hyphen (-) between the GUID and the ID.

In other words we need find in folder with FID 17A0, message with MID 502eb.

How determine folder name by FID:

Use PFDAVAdmin (https://www.microsoft.com/en-us/download/details.aspx?id=22427) to open required mailbox:

After you connected to mailbox:

Choose Tools -> Export Properties

In PropertyExportForm we need to select:

ptagFID: 0x67480014

and check, that PR_DISPLAY_NAME: 0x3001001E selected

All other property's checkboxes can be cleaned:

After folder's properties successfully exported

We can check output file:

As a result Calendar folder has FID 2-17A0 (Календарь 2-17A0)

Now we determined Folder name by FID.

How determine message by MID:

Use MrMAPI (https://mfcmapi.codeplex.com/) to export message MIDs:

As we know, that it is calendar folder we can use for default profile:

mrmapi.exe -folder 1 -mid >calendar-mids.txt

For other profile:

mrmapi.exe -folder 1 -mid -profile Outlook > calendar-mids.txt

of cause we can dump all messages MID's in mailbox by:

mrmapi.exe -mid >calendar-mids.txt

As a result you will get something like:

9-502eb R MeetingN

9-502ec R MeetingN+1

9-502ed R MeetingN+1

So our target - MeetingN

Now we know Folder and Message.

P.S.

You can also use MrMAPI to export FIDs: mrmapi -FID

IOS and "shifted appointments" or How Exchange stores appointment start and end time.

Khrebin Dmitry — Fri, 07 Dec 2012 04:48:00 GMT

An example of wrong meeting created by
iPhone and iPAD with IOS 5.01 and 5.1 in Moscow (+04:00 UTC without DTS) TZ

Meeting beginning:

Meeting end:

Everything fine.

Here how TZ stamp in “user friendly”
format, which showed in client:

also seems good

Here is TZ description:

Full text:

Time Zone Definition:

bMajorVersion = 0x02 (2)

bMinorVersion = 0x01 (1)

cbHeader = 0x0020 (32)

wReserved = 0x0002 (2)

cchKeyName = 0x000D (13)

szKeyName = Europe/Moscow

cRules = 0x0001 (1)

TZRule[0x0].bMajorVersion = 0x02 (2)

TZRule[0x0].bMinorVersion = 0x01 (1)

TZRule[0x0].wReserved = 0x003E (62)

TZRule[0x0].wTZRuleFlags = 0x0002 =
TZRULE_FLAG_EFFECTIVE_TZREG

TZRule[0x0].wYear = 0x0001 (1)

TZRule[0x0].X = cb: 14 lpb:
0100000001000000000000000000

TZRule[0x0].lBias = 0xFFFFFF4C (-180)

TZRule[0x0].lStandardBias = 0x00000000 (0)

TZRule[0x0].lDaylightBias = 0x00000000 (0)

TZRule[0x0].stStandardDate.wYear = 0x0 (0)

TZRule[0x0].stStandardDate.wMonth = 0x0 (0)

TZRule[0x0].stStandardDate.wDayOfWeek = 0x0
(0)

TZRule[0x0].stStandardDate.wDay = 0x0 (0)

TZRule[0x0].stStandardDate.wHour = 0x0 (0)

TZRule[0x0].stStandardDate.wMinute = 0x0
(0)

TZRule[0x0].stStandardDate.wSecond = 0x0
(0)

TZRule[0x0].stStandardDate.wMilliseconds =
0x0 (0)

TZRule[0x0].stDaylightDate.wYear = 0x0 (0)

TZRule[0x0].stDaylightDate.wMonth = 0x0 (0)

TZRule[0x0].stDaylightDate.wDayOfWeek = 0x0
(0)

TZRule[0x0].stDaylightDate.wDay = 0x0 (0)

TZRule[0x0].stDaylightDate.wHour = 0x0 (0)

TZRule[0x0].stDaylightDate.wMinute = 0x0
(0)

TZRule[0x0].stDaylightDate.wSecond = 0x0
(0)

TZRule[0x0].stDaylightDate.wMilliseconds =
0x0 (0)

Attribute description can be founded here:
https://msdn.microsoft.com/en-us/library/ee160657(v=exchg.80).aspx

This case related with:

lBias
(4 bytes): This field specifies the time zone's
offset in minutes from UTC.

iPhone stamp it as -180 from UTC (+03:00
UTC), instead of (+04:00 UTC)

Here is same attribute from meeting created
in Outlook 2010:

Time Zone Definition:

bMajorVersion = 0x02 (2)

bMinorVersion = 0x01 (1)

cbHeader = 0x0030 (48)

wReserved = 0x0002 (2)

cchKeyName = 0x0015 (21)

szKeyName = Russian Standard Time

cRules = 0x0002 (2)

It for 2010 year, where we still have DTS in Russia:

TZRule[0x0].bMajorVersion = 0x02 (2)

TZRule[0x0].bMinorVersion = 0x01 (1)

TZRule[0x0].wReserved = 0x003E (62)

TZRule[0x0].wTZRuleFlags = 0x0000 = 0x0

TZRule[0x0].wYear = 0x07DA (2010)

TZRule[0x0].X = cb: 14 lpb:
0100000001000000000000000000

TZRule[0x0].lBias = 0xFFFFFF4C (-180)

TZRule[0x0].lStandardBias = 0x00000000 (0)

TZRule[0x0].lDaylightBias = 0xFFFFFFC4
(-60)

Here we describe when we move arrows on our clocks:

TZRule[0x0].stStandardDate.wYear = 0x0 (0)

TZRule[0x0].stStandardDate.wMonth = 0xA
(10)

TZRule[0x0].stStandardDate.wDayOfWeek = 0x0
(0)

TZRule[0x0].stStandardDate.wDay = 0x5 (5)

TZRule[0x0].stStandardDate.wHour = 0x3 (3)

TZRule[0x0].stStandardDate.wMinute = 0x0
(0)

TZRule[0x0].stStandardDate.wSecond = 0x0
(0)

TZRule[0x0].stStandardDate.wMilliseconds =
0x0 (0)

TZRule[0x0].stDaylightDate.wYear = 0x0 (0)

TZRule[0x0].stDaylightDate.wMonth = 0x3 (3)

TZRule[0x0].stDaylightDate.wDayOfWeek = 0x0
(0)

TZRule[0x0].stDaylightDate.wDay = 0x5 (5)

TZRule[0x0].stDaylightDate.wHour = 0x2 (2)

TZRule[0x0].stDaylightDate.wMinute = 0x0
(0)

TZRule[0x0].stDaylightDate.wSecond = 0x0
(0)

TZRule[0x0].stDaylightDate.wMilliseconds =
0x0 (0)

Definition for year 2011 (not perfect, because we refused DST only in
summer so first month must have it? But
for 2012 it is OKey):

TZRule[0x1].bMajorVersion = 0x02 (2)

TZRule[0x1].bMinorVersion = 0x01 (1)

TZRule[0x1].wReserved = 0x003E (62)

TZRule[0x1].wTZRuleFlags = 0x0002 =
TZRULE_FLAG_EFFECTIVE_TZREG

TZRule[0x1].wYear = 0x07DB (2011)

TZRule[0x1].X = cb: 14 lpb:
0100000001000000000000000000

TZRule[0x1].lBias = 0xFFFFFF10 (-240)

TZRule[0x1].lStandardBias = 0x00000000 (0)

TZRule[0x1].lDaylightBias = 0xFFFFFFC4
(-60)

TZRule[0x1].stStandardDate.wYear = 0x0 (0)

TZRule[0x1].stStandardDate.wMonth = 0x0 (0)

TZRule[0x1].stStandardDate.wDayOfWeek = 0x0
(0)

TZRule[0x1].stStandardDate.wDay = 0x0 (0)

TZRule[0x1].stStandardDate.wHour = 0x0 (0)

TZRule[0x1].stStandardDate.wMinute = 0x0
(0)

TZRule[0x1].stStandardDate.wSecond = 0x0
(0)

TZRule[0x1].stStandardDate.wMilliseconds =
0x0 (0)

TZRule[0x1].stDaylightDate.wYear = 0x0 (0)

TZRule[0x1].stDaylightDate.wMonth = 0x0 (0)

TZRule[0x1].stDaylightDate.wDayOfWeek = 0x0
(0)

TZRule[0x1].stDaylightDate.wDay = 0x0 (0)

TZRule[0x1].stDaylightDate.wHour = 0x0 (0)

TZRule[0x1].stDaylightDate.wMinute = 0x0
(0)

TZRule[0x1].stDaylightDate.wSecond = 0x0
(0)

TZRule[0x1].stDaylightDate.wMilliseconds =
0x0 (0)

So we can see -240 minutes from UTC instead of -180 in previous one.

Resolution:

1 Involve vendor to resolve this issue

2 Use Abu-Dhabi TZ, which is +04:00 for
years w/o DTS