usually this should be resolved by following KB https://support.microsoft.com/kb/2579075

in some cases the above KB article doesn't solve the issue so we have to add the permissions at a higher level using dsacls as below

in my case I found the below permissions missing

Domain Name\Exchange Servers    SPECIAL ACCESS for msExchActiveSyncDevices   <Inherited from parent>
                                     

                                      CREATE CHILD
                                      DELETE CHILD
                                      LIST CONTENTS

Dsacls “dc=contoso,dc=com” /I:S /G “Contoso\exchange servers:CCDC;msexchActiveSyncDevices;user”
Dsacls “dc=contoso,dc=com” /I:S /G “Contoso\exchange servers:LC;;user”

 after that the user should be able to sync his mobile

in this Blog I will only cover the mailbox role and will do simulate a mailbox servers failure in production site that requires a full site switchover, there are other steps you need to do to activate the other server roles that are not mentioned here 

we have the below Organization that contains two Sites

CLT site (production)

ADA-CLT-MBX, ADA-CLT-MBX02, ADA-CLT-HC

SEA site (DR)

ADA-SEA-MBX, ADA-SEA-HC

we are going to deploy a DAG on three Mailbox servers and learn how to do site switchover

first we need to create a new domain called  Adatum.com

place one DC in CLT site and another one in SEA site

install exchange on all servers in both CLT and SEA sites

follow this blog to implement DAG

 https://blogs.technet.com/b/winde76/archive/2011/03/23/step-by-step-create-a-database-availability-group-dag.aspx

No DAC Enabled

to test our DR site switch over we need to do the following steps

• simulate the production site power failure by switching off all exchange mailbox servers in main site
• Stop the Cluster service on each DAG member in the second data-center by running the following command on each member:

net stop clussvc

• On a DAG member in the second data-center, force a quorum start of the Cluster service by running the following command:

net start clussvc /forcequorum

• Open the Fail-over Cluster Management tool and connect to the DAG's underlying cluster. Expand the cluster, and then expand Nodes. Right-click each node in the primary data-center, select More Actions, and then select Evict.

• activate mailbox servers in DR site

The quorum must be modified based on the number of DAG members in the second data-center.

• If there's an odd number of DAG members, change the DAG quorum model from a Node a File Share Majority to a Node Majority quorum by running the following command:
  
  cluster <DAGName> /quorum /nodemajority

• If there's an even number of DAG members, reconfigure the witness server and directory by running the following command in the Exchange Management Shell:

Set-DatabaseAvailabilityGroup <DAGName> -WitnessServer <ServerName>

in our scenario we have odd number so we will use

cluster loayaldag /quorum /nodemajority

net start clussvc

•  Perform server switch-overs to activate the mailbox databases in the DAG by running the following command 

• start all mailbox servers

• remove the copies from the Primary site

• remove the servers from the DAG

• add the servers back to the DAG either using EMC or Add-DatabaseAvailabilityGroupServer 

• add database copies again

• first we need to enable DAC mode by running

• simulate the failure shutdown the mailbox servers in the primary site

•  The DAG members in the primary data-center must be marked as stopped in the primary data-center. Stopped is a state of Active Manager that prevents databases from mounting, and Active Manager on each server in the failed data-center is put into this state by using the stop-DatabaseAvailabilityGoup cmdlet from the Primary site servers, If the Mailbox server is unavailable but Active Directory is operating in the primary data-center, the Stop-DatabaseAvailabilityGroup command with the ConfigurationOnly parameter must be run against all servers in this state in the primary data-center

• to complete activation of the mailbox servers in the second data-center are as follows

•  The databases can now be activated. Depending on the specific configuration used by the organization, this may not be automatic. If the servers in the standby datacenter have an activation blocked setting, the system won't do an automatic failover from the primary datacenter to the standby datacenter of any database. If no failover restrictions are present for any of the database copies in the standby datacenter, the system will activate copies in the second datacenter assuming they are healthy. If databases are configured with an activation blocked setting that requires explicit manual action, there are two choices for action:

 in our scenario there is no block so we will just go for activation

get-mailboxdatabase | Move-ActiveMailboxDatabase -ActivateOnServer ada-sea-mbx -SkipActiveCopyChecks -SkipClientExperienceChecks -SkipHealthChecks -SkipLagChecks  -MountDialOverride:besteffort

• after we fix the issues on the servers in the clt site we need to restore the service and add them back to DAG

• First you can reincorporate the DAG members in the primary site by using the Start-DatabaseAvailabilityGroup cmdlet. Then, to make sure that the proper quorum model is being used by the DAG, run the Set-DatabaseAvailabilityGroup cmdlet against the DAG without specifying any parameters.

  Start-DatabaseAvailabilityGroup -Identity loayaldag -ActiveDirectorySite clt

Set-DatabaseAvailabilityGroup

• After the Mailbox servers in the primary data-center have been incorporated into the DAG, they will need some time to synchronize their database copies. Depending on the nature of the failure, the length of the outage, and actions taken by an administrator during the outage, this may require reseeding the database copies

 hope that the above is clear and straightforward

You may get the following error when you run Get-ReceiveConnector cmdlet from
Exchange Management Shell

The Exchange group with GUID
"3f965b9c-f167-4b4a-936c-b8efb19c4784" was not found. This group was
automatically created during Exchange Setup, but has been subsequently
removed.


More Information:
=================

This typically
happens if one or more Exchange Security groups are missing. There are 6
Exchange security groups . Exchange Organization Administrators, Exchange
Public Folder Administrators, Exchange Recipient Administrators, Exchange
Servers, Exchange View-Only Administrators and ExchangeLegacyInterop.

In this situation Exchange security group "ExchangeLegacyInterop" is missing. Running
the Exchange Setup /PrepareAD may fail as well with the following
error

[13/05/2009 10:01:40 PM] [2] [ERROR] The well-known object entry
B:32:9C5B963F67F14A4B936CB8EFB19C4784:CN=ExchangeLegacyInterop\0ADEL:90d1a283-ed42-4ddf-8402-f0dbef0290b2,CN=Deleted
Objects,DC=CAO,DC=local of the otherWellKnownObjects attribute on container
object CN=Configuration,DC=CAO,DC=local points to an invalid DN or a deleted
object. Please remove the entry and rerun the task.

Resolution:
=================

You cannot delete the invalid DN value
from ADSIEDIT.msc so you have to use LDAP. Once you have deleted the invalid DN
reference, you need to run the PrepareAD command from the Exchange setup to
create the missing Security Group.

1.) Go to Start -> Run -Type
LDP.exe -> click OK
2.) Click Connection -> Connect and then click
OK
3.) Click Connection -> Bind and then click OK
4.) Click View ->
Tree and for the BaseDN select CN=Configuration,DC=DomainName,DC=local and click
OK
5.) In the left-pane expand the Configuration partition -> Services
-> Microsoft Exchange
6.) Right-click the object Microsoft Exchange and
then click Modify
7.) In the Attribute box type otherWellKnownObjects
8.) In the Values box
type
B:32:9C5B963F67F14A4B936CB8EFB19C4784:CN=ExchangeLegacyInterop\0ADEL:90d1a283-ed42-4ddf-8402-
f0dbef0290b2,CN=Deleted Objects,DC=CAO,DC=local (The invalid DN reference you
want to delete)
9.) Select the Delete radio button
10.) Click the Enter
button
11.) The Entry List text box will populate with [Delete] and the
selected attribute and its value in steps 7 & 8.
12.) Click on the Run
button

 if you by mistake cleared the otherwellknowobjects  value above and you  run setup /prepareAD it will create duplicate security group

what actually happens here

when we run setup /prepareAD, setup will go and check the contents of OtherWellKnownObjects attribute on Microsoft exchange object to see what security groups are existing, so it will find nothing and when it tries to create the security groups it will find that those groups exists in AD so it adds number one after each security group name

to resolve this issue the shortest way is to delete all exchange security groups, clear the OtherWellKnownObjects attribute and run exchange setup /prepareAD again

Connecting to remote server failed with the following error
message: The WinRM client cannot process the request. It cannot determine the
content type of the HTTP response from the destination computer. The content
type is absent or invalid. For more information, see the
about_Remote_Troubleshooting Help topic.

looking at the application log we can see the below warning

 Log Name:      Application

Source:        ASP.NET
2.0.50727.0

Date:         
3/11/2012 11:22:04 AM

Event ID:      1310

Task Category: Web Event

Level:         Warning

Keywords:      Classic

User:          N/A

Computer:     
ABC.XYZ.LOCAL

Description:

Event code: 3008

Event message: A configuration error has occurred.

Event time: 3/11/2012 11:22:04 AM

Event time (UTC): 3/11/2012 8:22:04 AM

Event ID: c21330b3c7e64e37938fefc93278b3b0

Event sequence: 2

Event occurrence: 2

Event detail code: 0

Application information:

    Application
domain: /LM/W3SVC/1/ROOT/PowerShell-1-129759272693526091

    Trust level: Full

    Application
Virtual Path: /PowerShell

    Application Path:
D:\Program Files\Microsoft\Exchange Server\V14\ClientAccess\PowerShell\

    Machine name:
ABC

Process information:

    Process ID: 7192

    Process name:
w3wp.exe

    Account name: NT
AUTHORITY\SYSTEM

Exception information:

    Exception type:
ConfigurationErrorsException

    Exception message:
Could not load file or assembly
'Microsoft.Exchange.Configuration.RedirectionModule, Version=14.0.0.0,
Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies.
The system cannot find the file specified. (D:\Program Files\Microsoft\Exchange
Server\V14\ClientAccess\PowerShell\web.config line 42)

Request information:

    Request URL:
https://ABC.XYZ.LOCAL/PowerShell?PSVersion=2.0

    Request path:
/PowerShell

    User: 

    Is authenticated:
False

    Authentication
Type: 

    Thread account
name: NT AUTHORITY\SYSTEM

Thread information:

    Thread ID: 7

    Thread account
name: NT AUTHORITY\SYSTEM

    Is impersonating:
False

    Stack trace:    at
System.Web.Configuration.ConfigUtil.GetType(String typeName, String
propertyName, ConfigurationElement configElement, XmlNode node, Boolean
checkAptcaBit, Boolean ignoreCase)

   at
System.Web.Configuration.Common.ModulesEntry..ctor(String

looking at the above warning looks like something missed up in the web.config under power shell directory

if we edit the web.config we can see a variable called %ExchangeInstallDir% but that variable doesn't exist in the environment variable path, even after we add it still the issue is the same

so what i did here is replace this variable with the physical path as below

open web.config

Replace all "file:///%ExchangeInstallDir%" with "file:///D:\Program Files\Microsoft\Exchange Server\" where D is the drive where Exchange is installed

Save the file

iisreset

now EMC and EMS should open properly

the issue was some users are not able to receive e mails from certain domains, when we check message tracking, no trace for the message, we enabled pipeline tracing and also no trace for the message, if the sender sends an e mail to the user and another few users in his organization the message is received for all users except the problematic user and the odd thing is that the user is not in the header of the message which means something is filtering the user before it even arrives at exchange server, we checked customer network devices and even we replaced some appliances with no luck, then we found that what is common between the problematic users are all having black berry accounts

and that's when we found that the messages were going to black berry instead of exchange server

Suppose you have a user with a recipient limit of 5000 and intermittently when he sends an e mail to a certain list of users he got an NDR similar to this

This message wasn't delivered to anyone because there are too many recipients. The limit is 50. This message has 216 recipients

but the user has a recipient limit of 5000, so what is special in this message

checking the message tracking i can see the same message failed also with ambiguous address, looking at the recipients in that message i could find an address which doesn't have an E mail and it displays as 'user name', so this looks like a bad address, Mail tips also shows that its invalid address

removing the address from the recipient list will solve the issue and bingo message is sent

• unique name for the DAG of up to 15 characters one or more IP addresses (either IPv4 or both IPv4 and IPv6) to the DAG, must be on the subnet intended for the MAPI network and must be available for use
• witness server and witness directory, recommended to be on HUB server
• one or more exchange 2010 mailbox server roles installed

in this scenario we have single Domain controller with exchange servers roles installed as below

DAG-CASh01: CAS and HUB server with single network card (192.168.1.4 /24)

DAG-MBX01 : Mailbox server with two network cards (192.168.1.2 /24) for MAPI and (11.11.11.1 /8) for replication

DAG-MBX02: Mailbox server with two network cards (192.168.1.6 /24) for MAPI and (11.11.11.2 /8) for replication

DC01 : domain controller with single network card (192.168.1.1 /24)

Database Availability Group Name : LabDAG

Database Availability Group IP Address: 192.168.1.10

Creating the DAG (EMC)

1. In the console tree, navigate to Organization Configuration > Mailbox.
2. In the action pane, click New Database Availability Group.
3. specify the DAG name

• notice that i left the Witness Server and Witness Directory check boxes cleared. In this scenario, the wizard will search for a Hub Transport server that doesn't have the Mailbox server role installed. It will automatically create the default directory and share on that Hub Transport server and use that server as the witness server.

Assigning DAG IP Address

now the DAG is ready we need to assign an IP Address

set-DatabaseAvailabilityGroup -Identity LABDAG -DatabaseAvailabilityGroupIpAddresses 192.168.1.10

 now DAG is ready and next step is to add a new member to DAG

Adding a new member to DAG (EMC)

• In the console tree, navigate to Organization Configuration > Mailbox.
• In the result pane, click the Database Availability Group tab, right-click the DAG you want to manage, and then click Manage Database Availability Group Membership
• On the Manage Database Availability Group Membership page click Add, add the first exchange mailbox server from the list, Manage and Finish

now we have DAG with one member and notice that net DAG networks created automatically

 now we can add other members by repeating the steps above in the add member section

• In the console tree, navigate to Organization Configuration > Mailbox.

 On the Add Mailbox Database Copy page, complete the following fields.

• Mailbox database name This read-only box displays the name of the database for which you are going to make a copy.
• Server name Click Browse to open the Select Mailbox Server dialog box. Use this dialog box to select the Mailbox server that will host the copy of the mailbox database, and then click OK.

 References

https://technet.microsoft.com/en-us/library/dd351172.aspx

https://technet.microsoft.com/en-us/library/dd351278.aspx

https://technet.microsoft.com/en-us/library/dd298080.aspx

environment

first installed exchange 2007 or exchange 2010 not CAS array member

2 exchange 2010 CAS array members NLB

When the first CAS is shut down, Ol2k7 Client try to connect to its SCP to retrieve the Autodiscover settings.

Users get a pop-up authentication window. If they hit cancel, then Outlook connects to the next available SCP

Outlook always tries to contact the SCP of the first installed CAS Server, which is not member of the CAS array

to resolve this issue

Change SCP of First CAS to match the FQDN of the CAS Array using command Set-ClientAccessServer –AutodiscoverServiceInternalUri