By default, clients that attempt to negotiate authentication will fail Kerberos authentication when a CAS array is configured thus falling back to NTLM. The reason Kerberos authentication fails is because the authentication request is sent using the FQDN of the CAS array which does not match the SPN value on the CAS servers. To address this issue we must create an appropriate SPN value for the CAS array within Active Directory. We also cannot configure the same SPN value on multiple objects in Active Directory. To work around this we created a method called Alternate Service Account (ASA). The ASA can be either a computer account or user account. Then we can associate the SPNs we want for the CAS array to the ASA.

Note. We can use one ASA for all CAS arrays or make an ASA for each CAS array.

Creating the alternate service account (ASA)

Open Active Directory Users and Computers and create the ASA. In this example we will create a computer account.

The account name does not matter as long is its valid but consider making the name self explanatory.

Fill in a description. (This is not required but should to prevent some-one from deleting the account)

Setting the correct SPNs to the ASA

Confirm the FQDN of the CAS array with the following command in Exchange Management Shell.

Get-ClientAccessArray 

Now we need to set the SPNs of the CAS array Autodiscover and EWS. Some MAPI applications that interact with Exchange may need to use the Array’s short name or NetBIOS. We will set NetBIOS and FQDN for each SPN.
In this example the commands will be as follows. (If a computer account is being used, a $ must be after the name you have assigned.)

Note. At this point new authentication attempts to the above DNS names will fail until the ASA is assigned to the CAS servers.At this point we could back out by simply running the following command for each SPN. setspn –d {spn} {account}

Setting the ASA to the CAS servers

From here we will want to assign the ASA to our CAS servers so that clients can start authenticating to the CAS array.
Go to the Scripts directory located in the Exchange install directory (by default it is C:\Program Files\Microsoft\Exchange Server\V14\Scripts\)

Run the following command in Exchange Management Shell.

.\RollAlternateserviceAccountPassword.ps1 –ToArrayMembers {CAS array name}-GenerateNewPasswordFor {Domain}\{ASA}" –Verbose

The script will ask if we want to change the password on the ASA.In this case we will say Yes.

The final output will tell us if we succeeded or failed on any part of the script.

Confirm that CAS is accepting Kerberos

Now we want to verify that the CAS servers can accept Kerberos requests.
Check in the Address book logs to see if the authentication method is set to Kerberos. (Default location is C:\Program Files\Microsoft\Exchange server\v14\Logging\AddressBook Service)

2011-03-11T20:00:12.000Z,9,0,/o=First Organization/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=Administrator,,192.168.0.123,proseware-hc1,ncacn_ip_tcp,Bind,,6,,,Kerberos,

OR

Since this blog is geared to exchange and networking. Take a netmon and look to see if we are using a Kerberos Token.

Side notes

If you ever add an additional node to the CAS array you can use the following command to copy the ASA credentials over to the new server without changing the password on the ASA.

.\RollAlternateServiceAccountPassword.ps1 -CopyFrom ServerA -ToSpecificServers ServerB –Verbose

To find the CAS servers that have been configured with an ASA run the following command.

Get-ClientAccessServer -IncludeAlternateServiceAccountCredentialstatus |Fl Name, AlternateServiceAccountConfiguration

Related Articles

Configuring Kerberos Authentication for Load-Balanced Client Access Servers

Using Kerberos with a Client Access Server Array or a Load-Balancing Solution

Using the RollAlternateserviceAccountCredential.ps1 Script in the Shell

When Outlook 2007/2010 is setup with an Exchange profile by default it will enable RPC encryption. This prevents devices or support from reviewing the raw remote operation (ROP) sent between the client and server. Normally this is not a big deal but in some support cases vital information can be given by reviewing the ROPs.

How to disable MAPI/RPC encryption on the client. Outlook 2003/2007/2010.

1. In the Windows Control Panel, double click Mail.
2. Choose Email Accounts…
3. Highlight the Exchange account and click Change.
4. Click More Settings…
5. Click on the Security Tab.
6. Ensure "Encrypt data between Microsoft Office Outlook and Microsoft Exchange." is Unchecked.
   If the option is checked and greyed out the there is most likely a GPO applying this setting.

Now the Exchange server must allow unencrypted clients to connect. By default Exchange 2003/2007 allows unencrypted clients but Exchange 2010 does not. Exchange 2010 requires encryption.

How to check and disable the requirement of MAPI encryption on Exchange 2010.

In the Exchange Management Shell run the following commands
Get-RPCClientAccess | ft Server, EncryptionRequired
Note the setting of the servers should you want to change them back
Get-RPCClientAccess | Set-RPCClientAccess –EncryptionRequired:$False

Note. This change does not require the restarting of any services and will not cause any current clients to disconnect. Clients with MAPI encryption enabled will still be able to connect. Must be done to all CAS servers in the array and the Public Folder server.

How to check and disable the requirement of MAPI encryption on Exchange 2007.

In the Exchange Management Shell run the following commands
Get-MailboxServer | ft Name, MapiEncryptionRequired
Get-MailboxServer | Set-MailboxServer -MapiEncryptionRequired:$false

Now that the above has been done we can start taking unencrypted network trace.
Template: How to take Network traces between clients and exchange servers.

To confirm that the client has been set for unencrypted traffic you can open the trace in netmon and review the following.

Open the capture with netmon and find a MAIPI response. Review the Frame Details and drill down to the AuthVerifier. If you see the below then the client was not setup correctly.

Frame: Number = 4100, Captured Frame Length = 134, MediaType = ETHERNET
+ Ethernet: Etype = Internet IP (IPv4),DestinationAddress:[00-00-00-00-00-00],SourceAddress:[00-00-00-00-00-00]
+ Ipv4: Src = 10.10.10.10 Dest = 10.20.20.20, Next Protocol = TCP, Packet ID = 14063, Total IP Length = 120
+ Tcp: Flags=...AP..., SrcPort=7575, DstPort=54125, PayloadLen=80, Seq=2622971141 - 2622971221, Ack=3837493092, Win=255 (scale factor 0x8) = 65280
- MSRPC: c/o Response: Exchange Server STORE EMSMDB Interface {A4F1DB00-CA47-1067-B31F-00DD010662DA}  Call=0x8  Context=0x0  Hint=0x18  Cancels=0x0 Warning: Octets trailer appends to authentication token
  - Response:
     RpcVers: 5 (0x5)
     RpcVersMinor: 0 (0x0)
     PType: 0x02 - Response
   + PfcFlags: 3 (0x3)
   + PackedDrep: 0x10
     FragLength: 80 (0x50)
     AuthLength: 16 (0x10)
     CallId: 8 (0x8)
     AllocHint: 24 (0x18)
     PContId: 0 (0x0)
     CancelCount: 0 (0x0)
     Rsvd1: 0 (0x0)
   - PDUBody: 24 bytes
    - StubData: 0x1
       MSRPCPayload: Binary Large Object (24 Bytes)
   - AuthVerifier:
      AuthPad: Binary Large Object (8 Bytes)
      AuthType: RPC_C_AUTHN_WINNT - NTLM authentication will be used.
      AuthLevel: dce_c_authn_level_pkt_privacy - This level offers the dce_c_authn_level_pkt_integrity services plus privacy (encryption) of stub call arguments only. All run-time and lower-layer headers are still transmitted in clear text.
      AuthPadLength: 8 (0x8)
      AuthReserved: 0 (0x0)
      AuthContextId: 0 (0x0)
    + AuthValue:

If Encryption is enabled again on the servers go back to the outlook profile an re-enable "Encrypt data between Microsoft Office Outlook and Microsoft Exchange." otherwise the client will not be able to connect again.

additional links

https://msdn.microsoft.com/en-us/library/ee200574%28v=EXCHG.80%29.aspx

remote operation (ROP): An operation that can be invoked against a server. Each ROP represents an action, such as Delete, Send, or Query. It is encoded on the wire by a ROP number, followed by parameters that are specific to the operation. More than one ROP can be packed into a single RPC request or response.

https://msdn.microsoft.com/en-us/library/aa373553%28v=vs.85%29.aspx

RPC_C_AUTHN_LEVEL_PKT_PRIVACY

Includes all previous levels, and ensures clear text data can only be seen by the sender and the receiver. In the local case, this involves using a secure channel. In the remote case, this involves encrypting the argument value of each remote procedure call.

In the last couple of years we have noticed that due to the complexity of companies and networks support must collect more and more network traces. This is to help people collect the correct data the first time.

You will need a network tap device or software that can collect network traffic. There are two widely used capturing software’s: Microsoft Network Monitor (netmon) or Wireshark.

Below is an example using netmon between a windows XP/Vista/7 client and Exchange 2003/2007.

1. Depending on how exchange is setup you will need to know where the effected user’s mailbox is homed and if that server is a cluster. The client will always attempt to connect to the Active node were store is running.
2. Install netmon on your client. (if not already installed)
Note: I do not give the link for downloading netmon but go to https://download.microsoft.com and search “netmon” the latest version will come up. Also download the correct version depending on if the OS is 32bit or 64bit.
3. Install netmon on your Exchange server. (if not already installed)
Note: if you are using ISCSI targets over Ethernet, installing a network capturing software can cause a “blip” on the wire because it’s adding a driver to the network stack. This could cause loss of disks for a second and worse case cause database corruption. If this is an issue, install netmon on your passive node first and then failover to that node. If you have JBOD or Fiber Channel this is a non-issue.
4. On the client be sure that outlook is closed and the process is not still running in Task Manager.
5. At this point go into a command prompt Start > Run type CMD click OK.
6. In to command prompt you will have to navigate to the install folder netmon is installed. Type cd "c:\Program Files\Microsoft Network Monitor 3" (default install location)
7. To start the network trace type nmcap.exe /network * /capture /file ExchangeTrace-client.chn:100M
Note: This will allow netmon to collect a network trace from all adapters and save them in 100 MB sequential files. This will make it easier to upload and review.
8. Go to the Exchange server and do Steps 5. 6 and 7. Change the name of the file to ExchangeTrace-server.chn:100M
9. On your client start outlook and reproduce the issue.
10. Now we need to stop the network traces. In the command prompt press CTRL+C. Remember to do it on both client and server.
11. Note the actions taken during the reproduction of the issue with time stamps. This allows the person reviewing the network traces to better understand why specific actions happened in the trace.

For Exchange 2010 Outlook clients connect to the Client Access Server for Mailbox requests and the Mailbox server for Public Folder requests. Also, with CASarray you will need to capture on all Client Access Servers or use a Host file record to bypass the load balancer and have a 1:1 network trace.

Notes to the above content

As we saw above, the command to start a capture on all adapters and make 100MB sequential files.

nmcap.exe /Network * /Capture /CaptureProcesses /File ExchangeTrace.chn:100M

Now here is how to do the same but add a filter since maybe we know we only want traffic to and from the server and client but want to drop everything else from the trace.

In the below example the client IP is 10.10.50.40 and the server IP is 10.1.75.20

nmcap.exe /Network * /Capture "(ipv4.Address == 10.10.50.40 and ipv4.Address == 10.1.75.20)" /CaptureProcesses /File ExchangeTrace.chn:100M

If you are familiar with netmon filtering you can see what is in the parentheses is basic netmon display filtering. You can do other filters like (tcp.port == 80) but be cautious using filtering. We may filter out other requests that could help understand why the issue is happening, such as Kerberos or LDAP requests and much more.

For high performance network capturing on an exchange server or heave load client we can use the following command.
nmcap.exe /Network * /Capture /File ExchangeTrace.chn:100M /MinDiskQuotaPercentage 10 /UseProfile 85D454AF-5EED-4d4d-8BAB-588631A33429 /DisableConversations

This command will stop the capture if the disk we are saving our output to reaches 10% free space and uses the Pure Capture profile thus not having to parse the packets as the are recoded.

Also consider reading Reducing Dropped Frames with Network Monitor 3.4

Below is how to take a network capture on an Apple client or a Linux/Unix client.(subject to change on TCPdump version)

In a terminal you can use TCPDump to collect network traffic and netmon is able to review the cap files created.

Apple OSX 10.5.x

Sudo tcpdump –t en0 –s 0 –C 100 –w ~/Desktop/netcap.cap

To stop tracing press Control+C

Apple OSX 10.6.x

sudo tcpdump -i en0 -s 0 -B 10485760 –C 100 -w ~/Desktop/netcap.pcap
To stop tracing press Control+C

Linux/Unix
tcpdump -i any -s 0 -B 10485760 –C 100 -w /home/{user}/netcap.pcap
To stop tracing press CTRL+C