So...for my latest scripting foray, I came up with a little script to do the following:

• Find all User accounts that have a SamAccountName that matches a particular filter, and that DO NOT have their passwords set to Never Expire (so we don't hit service accounts, etc)
  
• If the account is a member of a particular group, leave the mailbox intact, but set a particular retention policy on it (in this case, a 14 day policy)
  
• If the disabled account is NOT a member of the aforementioned group, remove the Exchange attributes from the AD account, or essentially, disable the mailbox
  
• And, that's pretty much it. I also logged all accounts that we touch to a file, and added a timer so I’d know how long it takes the script to run.

 For this particular application, the script was written for and run against an Exchange 2007 environment FROM the Exchange management shell. Theoretically it should work with other versions as well, but I have yet to test that scenario.

 Also, please note that the script requires the Active Directory Module to be imported, so it cannot be run from instances of Exchange 2007 that are running on Windows Server 2003.

 For my purposes, all user account names have a 6 digit number in them. To make sure I grab only those accounts, a regex filter was used against the SamAccountName attribute so that only those accounts are matched, which is what the "\d{6}" is. You can take out that portion, or update the filter to match your needs.

Hopefully this is a good example of how you can automate your more routine tasks. Enjoy!

Here is a snippet of the script. Please go HERE to download the full copy.. 

$stopwatch = New-Object system.Diagnostics.Stopwatch

$stopwatch.Start();

$date = Get-Date -format MMddyy

$LogFile = "c:\LogFile_$date.txt"

$grouppath = "CN=GG-Termed-Retain,DC=contoso,DC=com" #Change this group name to match your environment!!!

write-host "Please wait while I import the active directory module..."

Import-Module ActiveDirectory

start-sleep -s 15

$DisabledAccounts = get-mailbox -resultsize Unlimited –ErrorAction SilentlyContinue –WarningAction SilentlyContinue | where {($_.UserAccountControl -eq "AccountDisabled, NormalAccount") –and ($_.RecipientTypeDetails –ne “RoomMailbox”) –and ($_.SamAccountName –match “\d{6}”)}

$Count = $DisabledAccounts.Count

Thanks, Kris.

So, here's the scenario...

• Company A acquires Company B
• Company B has it's own Forest/Own Domain
• We need to get Company B's users onto Company A's Exchange servers ASAP
• Company B's client computers WILL REMAIN on Company B's domain (they are not yet ready to join the client computers to Company A's domain)

Using some of the techniques described here, we were able to get Outlook Autodiscover to correctly locate the mailboxes that had been created for these users in Company A's forest/domain. No trust was yet in place between the two domains, and while there was network connectivity between the two forests, it was a rather complex web of tunnels, firewalls, NAT's, etc. So, it was difficult to determine which paths and ports were open between servers in which forest.

Now comes the fun part :). The trust between the two forests was brought up and shortly thereafter we found that users from Forest B were having symptoms indicative of Autodiscover not working. In short, no free/busy, no ability to set out of office, and unable to autoconfigure a new Outlook profile. I found it odd that only users from Forest B were affected, and immediatly suspected the problem had something to do with the trust, but I had no proof.

IIS logs for this customer were quite large, so I began pulling one of those down while doing some testing from an Outlook client. I used the Test E-mail AutoConfiguration option from the below menu. To bring up this option, hold down the <ctrl> key while right-clicking the Outlook icon in the BOTTOM RIGHT of your screen. Click the "Test E-mail AutoConfiguration", as noted below:

Next, we uncheck the two "Guess" boxes, and I left the password option blank, like so:

Again, the client computers were connected to Forest B. Since the trust was now in place, they theoretically should be able to present their Forest B credentials while hitting Autodiscover in Forest A and get authenticated just fine. What happened instead was that the client received a 500 error, and the autodisover test errored out. A 500 error USUALLY indicates some type of server error, but again, this issue was affecting only clients in Forest B. Clients from Forest A could authenticate just fine.

A review of the IIS log clearly showed the 500 error, but it had an unusual substatus code with it. The error looked something like this:

500 0 2148074257

Turning on friendly http error messages yielded the same 500 error along with 0x80090311. This link will show you how to turn on friendly errors in IIS.

Using the err utility, similar to the one you can download from here, I translated this numerical error code into something more meaningful:

“No authority could be contacted for authentication”

Now we're getting somewhere!

If we check in IIS, we can see that the Autodiscover directory has the following authentication methods enabled:

The key in this particular case is "Windows Authentication". Since "Windows Authentication" is enabled, the clients from Forest B were attempting to pass their Windows credentials along to IIS on the CAS in Forest A. The CAS in Forest A knew that it "should" be able to authenticate these users, but for some reason it was unable to do so, and it threw the

500 0 2148074257, 0x80090311, “No authority could be contacted for authentication”

errors back at the client. At this point, Outlook simply continued with its autodiscover logic rather than trying Basic authentication.

What we wound up doing as a solution was to open communication between the subset of DC's that serviced the Exchange servers in Forest A, with DC's in Forest B. Once we did this, Integrated (NTLM in this case) authentication worked as expected and all was well.

RESOURCES:

• You can also use www.testexchangeconnectivity.com to do your testing.
• Log Parser Studio - the best way to parse unwieldly IIS logs!

Thanks to Steve Wesa and Kary Wall for their assistance with this issue!

By now, everyone SHOULD be familiar with Autodiscover and what it does. Essentially, Autodiscover does the following:

• Configures profile settings for Outlook 2007 and above clients as well as for various types of mobile devices
• Provides Outlook clients with the URL's they need to access Exchange Web Services and the OAB download URL. EWS provides free/busy and Out of Office services.

Autodiscover is a function of EXCHANGE. Outlook 2007 and above clients are coded in such a way that they can take advantage of Autodiscover. It all works together (or it should). For Outlook to take advantage of the new Autodiscover features in Exchange 2007 and above, Outlook must be told where to go, or who to talk to if you will, to get all those fancy new features. That's where the Outlook logic comes into play in these scenarios. The order of logic that Outlook uses when trying to figure out where to get that information from is as follows:

• SCP lookup - Outlook will get Autodiscover information from Active Directory. If that fails, Outlook begins it's "non-domain" connected logic (as I like to call it), and will go in order down this list
• HTTPS root domain query - Outlook, if not domain joined, uses the RIGHT HAND SIDE of the users SMTP address to do this query. So using the domain from my example, it will search for https://wingtiptoys.com/autodiscover/autodiscover.xml
• HTTPS Autodiscover domain query - If the above search yields no response, the next URL Outlook will try is https://autodiscover.wingtiptoys.com/autodiscover/autodiscover.xml
• HTTP redirect method
• SRV record query
• Local XML file
• cached URL in the Outlook profile (new for Outlook 2013)

There are some important points to remember so that you can determine what behavior to expect form Outlook. The first thing you need to know is:

• Is the Outlook client domain joined AND can it currently CONTACT a DC in the domain!

The highlighted portion of that statement is an important distinction. If you are running Outlook on a laptop and you disconnect the network cable from said laptop and then try to configure a new Outlook profile, for example, your Outlook client will use it's NON domain joined logic at that point, because it CANNOT CONTACT ACTIVE DIRECTORY!

Where this becomes important is what I'd call "one off", or "unusal" circumstances, such as the absorption of one company by another. I ran across two such scenarios recently which is what prompted me to create this blog post. In these types of cases, it may be necessary to more fully control or tweak the Outlook to Exchange Autodiscover connection logic so that users will have a more robust experience, at least until the are fully absorbed into their new environment.

In both recent cases, the client computers were connected to Domain A (the "old" company) but needed to connect to Exchange servers in Domain B (the "new" company). One case was complicated further by the presence of Exchange 2007 in BOTH Domain A AND Domain B. And we needed to "test" connecting users in Domain B back to mailboxes created for them in Domain A WITHOUT making any modifications to either environment.

Since my client (meaning the actual computer I'm testing this with) is domain joined, and I am logged in with a domain account, Outlook automatically fills in my Name and E-mail address. If I just click next, Outlook will begin is logic and first try to locate Autodiscover settings via SCP. If my domain joined client can currently contact AD, it will get Autodiscover settings from AD and proceed with trying to configure the profile to connect to my mailbox (assuming I have a mailbox in that domain, assuming Exchange is installed and properly configured in the WingTipToys domain, etc).

The problem in the scenarios I referenced earlier, is that the customer wanted the Outlook client to connect to a mailbox in a completely different forest/domain. While you can certainly use the "Manually configure server" option, there are a few other things you could do.

At minimum, we must change the email address that has been autopopulated for us to the one that has been assigned to us from the "other" domain. For example purposes, I'll be using kris@corkandale.com.

For my customer, we decided to use a local autodiscover.xml file to override Outlook's default behavior, so that we could test some things without having to modify anything in either forest/domain. Think of this as a type of "hosts" file for Outlook. What you will do is create a file called autodiscover.xml that will look like this:

<?xml version="1.0" encoding="utf-8"?>
<Autodiscover xmlns="https://schemas.microsoft.com/exchange/autodiscover/responseschema/2006">
  <Response xmlns="https://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a">
    <Account>
      <AccountType>email</AccountType>
      <Action>redirectUrl</Action>
      <RedirectUrl>https://autodiscover.corkandale.com/autodiscover/autodiscover.xml</RedirectUrl>
    </Account>
  </Response>
</Autodiscover>

Then you need to configure the client machine to query that XML file by adding the following registry key:

Tell Outlook to use a local xml file:

• for Outlook 2007:

HKCU\Software\Microsoft\Office\12.0\Outlook\Autodiscover

• for Outlook 2010:

HKCU\Software\Microsoft\Office\14.0\Outlook\Autodiscover

 STRING_value <your_namespace> = path to XML file

Or 15.0 for Outlook 2013

Where  <your namespace> would be the namespace that Outlook will be querying for (corkandale.com in my example), and path to xml is the path to file.

NOTE: My client will STILL have to be able to resolve autodiscover.corkandale.com to an IP address. So, be sure that DNS is properly configured!

Even with the xml file in place, we still found that there was a pretty significant delay in both profile autoconfiguration AND in free/busy lookups. Using information from https://support.microsoft.com/kb/2612922 we excluded the following per the article:

• SCP lookup

• HTTPS root domain query

Once this had been done, we found things moved along much more quickly. This allowed my customer to do some more testing before fully integrating clients from Forest B into their own Forest.

For my latest project, I decided to try to put together a list of various tweaks that I recommend for Enterprise environments that run Microsoft Exchange Server 2007 x64 or higher. Please remember that these are RECOMMENDATIONS, not hard and fast guidelines! YMMV, please test these yourself, etc. The main goal here was to consolidate these recommendations into one place, rather than having to visit various blogs and TechNet articles to piece them all together. 

Many of the settings described herein are the defaults that were in place for 32 bit (x32) systems (i.e., Windows 2000/2003).They were never increased in the x64 versions of the product. Proactively increasing these values can help insure that x64 servers are more fully utilized. 

Please visit my colleague Rhoderick Milne's blog post for a few additional recommendations!

 Windows: MaxTokenSize – this setting can affect users with large tokens (also known as "token bloat") . The default value is low, imo, and the issue can be very frustrating to troubleshoot. The following article recommends setting the default to 48,000: https://support.microsoft.com/kb/938118, https://blogs.technet.com/b/surama/archive/2009/04/06/kerberos-authentication-problem-with-active-directory.aspx

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\Kerberos\Parameters
Entry: MaxTokenSize
Data type: REG_DWORD
Value: 48000

NOTE: The default for this value in Windows 2012 IS 48000. The maximum value is 65535 - https://support.microsoft.com/kb/938118

IIS: MaxFieldLength and MaxRequestBytes – these settings control the maximum header length and maximum number of bytes that are allowed in http headers. Users with large tokens can experience logon failures if the defaults are left in place. This article discusses those concerns: https://support.microsoft.com/kb/2020943

From the above referenced KB:

NOTE: If MaxFieldLength is configured to its maximum value of 64KB, then the MaxTokenSize registry value should be set to 3/4 * 64 = 48KB.  For more information on the MaxTokenSize setting, please see the Microsoft knowledge base article KB327825. Note that many of settings detailed in this post are interrelated.

Article https://support.microsoft.com/kb/2491354 for Exchange server details a free/busy issue that can be caused by low/default values for these settings, and reccommends setting them to their maxiumum allowed values of 65534 for MaxFieldLength and 16777216 for MaxRequestBytes

IIS: Queue Length for IIS application pools - I recommend changing the queue length value to at least 5000 (which is now the default) for all Exchange related application pools (Found in Advanced Settings on the properties of the Application Pools themselves). If you are still running Exchange 2007, your Queue Length is likely still at 1000. For example, if your application has its own App pool (such as OWAAppPool), the Advanced Settings of that App pool are where you would make this change. I have seen customers set this value higher (8000 or so) with no ill effects. PLEASE TEST IN YOUR ENVIRONMENT BEFORE MAKING DRASTIC CHANGES!

The following article describes the effects on Lync of leaving the default value of 1000 in place - https://technet.microsoft.com/en-us/library/dd441171(v=office.13).aspx

 To set the Queue Length for ALL app pools via appcmd, the syntax is as follows:

“%systemroot%\system32\inetsrv\appcmd.exe set config -section:system.applicationHost/applicationPools /applicationPoolDefaults.queueLength:"5000"  /commit:apphost”

 - Many thanks to Finbar Ryan for contributing this syntax!

Windows: MaxConCurrentAPI (On DC’s) – this setting affects certain types of authentication traffic (particularly NTLM) and is described here: https://support.microsoft.com/kb/975363.

The maximum value on either 2003 OR 2008 DC’s of any version (without the hotfix mentioned in the article) is 10, and I see no reason at all not to set ALL DC’s to this value, rather than leaving them at the default of 1. Please READ the article before making this change.

• Start Registry Editor.
• Locate the following registry subkey:
  HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Netlogon\Parameters
• Create the following registry entry:
  
  Name:  MaxConcurrentApi
  Type:  REG_DWORD
  Value:Set the value to the larger number, which you tested (any number greater than the default value).
• At a command prompt, run net stop netlogon, and then run net start netlogon.

The MaxConcurrentApi SDP is available to help troubleshoot MaxConcurrentApi bottlenecks: https://support.microsoft.com/kb/2709071

IIS: ASP.NET Request Queue Limits - (requestQueueLimit and appConcurrentRequestLimit)- Changing the request queue limit to 65,535K is known to resolve an issue that primarily exhibits itself on Exchange 2010 sp3 and above. However, there are other circumstances where changing this limit could be beneficial, and is another example of something that could be proactively changed before any box is put into production. https://technet.microsoft.com/en-us/library/dd425294(v=office.13).aspx (Lync article) says to set the limit for requestQueueLimit to 15,000. However, testing by the Exchange team has validated that a limit of 65,535 poses no problems in most circumstances.

As it specifically relates to Exchange, a change was made in the way Outlook Anywhere in SP3 and above traffic is processed that can cause it to bottleneck in ASP. This is typically seen only in very large environments, but I included this section anyway (just for thoroughness). In coming versions of Exchange, the rpc/http configurator will automatically set the requestQueueLimit to 65535 but ONLY for the rpc virtual directory. Thus, this it will be set not at the root level (as we are doing below), but at the rpc virtual directory level, so you will find the change in the web.config file specifc to the rpc virtual directory. See my IIS7 primer for more information about config files: https://blogs.technet.com/b/kristinw/archive/2012/11/28/iis7-primer-for-non-iis-admins-originally-written-for-exchange-engineers.aspx

To modify the values, do the following (copied from the above referenced TechNet article):

PLEASE MAKE A BACKUP OF THE CURRENT FILES BEFORE MAKING ANY CHANGES!!

1. Click Start and then click Run.

2. In the Run dialog box, type notepad %systemroot%\Microsoft.Net\Framework64\v2.0.50727\CONFIG\machine.config, and then click OK.

3. Locate the processModel element that looks like this: <processModel autoConfig="true" />

4. Replace the processModel element with the following value: <processModel enable="true" requestQueueLimit="65535" />

5. Save and close the Machine.config file

6. The above settings modify the global machine.config file. please notes that the below appConcurrentRequestLimit and the above requestQueueLimit are TWO DIFFERENT SETTINGS!

7. Now to edit the ApplicationHost.Config, in the Run dialog box, type %systemroot%\system32\inetsrv\appcmd.exe set config -section:System.Webserver/serverRuntime -appConcurrentRequestLimit:65535 -commitpath:APPHOST

1. If you have problems running the above command (some people have reported errors trying to run the command in virtual environments), you can manually make this change by locating the ApplicationHost.Config file in %systemroot%\system32\inetsrv\config directory. (You can also check this file to verify the change)

2. After locating the file AND MAKING A BACKUP COPY, open the file and search for serverRuntime until you find the one that looks like this: <serverRuntime />, located just underneath </security>.

3. Change the line to <serverRuntime appConcurrentRequestLimit="65535" />

4. Save and close the file.

NOTE: You are likely to hit other bottlenecks (notably port exhaustion) long before you ever hit the 65535 value noted in this post. This value CAN be set higher if you have configured port scaling.

PLEASE BE SURE THAT YOUR DO NOT FAT FINGER ANYTHING IN THE Machine.Config or ApplicationHost.config file!!! Bad things will happen if you do!

IIS - Speeding up Integrated Authentication

The following is courtesy of Greg Askew. The logic here is that certain web services within Exchange primarily use integrated authentication (internal OWA, EWS, anything OTHER than basic or anonymous auth). An easy way to determine what type of authentication a particular web service is using is to ask this question: upon initially hitting the site or service, does the get prompted for a username/password? If the answer is yes (this includes mobile devices that 'store' such information), then its probably integrated authentication of some type.

If integrated authentication is used extensively (NTLM or Kerberos) and there are lots of connections, IIS can be configured to authenticate ONLY the first request by using the following command:

appcmd set config /section:windowsAuthentication /authPersistNonNTLM:true

You may experience slow performance when you use Integrated Windows authentication together with the Kerberos authentication protocol in IIS 7.0 - https://support.microsoft.com/kb/954873

Kerberos authPersistNonNTLM authentication, request based vs. session based authentication - https://blogs.msdn.com/b/benjaminperkins/archive/2011/10/31/kerberos-authpersistnonntlm-authentication-request-based-vs-session-based-authentication.aspx

Many thanks to Casey Spiller, Steve Wesa, Chris Korff, David Santamaria, Finbar Ryan, Konstantin Papadakis and Greg Askew for contributing to these recommendations.

While it is documented that Windows devices only "support" certain policy settings:

https://support.microsoft.com/kb/2823900

https://support.microsoft.com/kb/2464593/en-us

There are SOME settings that can be configured that will prevent these devices from syncing AT ALL.

These settings are as follows:

-MaxCalendarAgeFilter
-MaxEmailAgeFilter
-AllowStorageCard
-WSSAccessEnabled
-UNCAccessEnabled

In my testing, I found that ANY of these settings being set to ANYTHING other than the values specified below will result in sync failure. I also included a sample of how I set the values using Exchange Management Shell:

Set-ActiveSyncMailboxPolicy $YourPolicy -MaxCalendarAgeFilter All
Set-ActiveSyncMailboxPolicy $YourPolicy -MaxEmailAgeFilter All
Set-ActivesSyncMailboxPolicy $YourPolicy -AllowStorageCard $true
Set-ActivesSyncMailboxPolicy $YourPolicy -WSSAccessEnabled $false
Set-ActivesSyncMailboxPolicy $YourPolicy -UNCAccessEnabled $false 

Here is the final, working policy that I came up with:

To APPLY the test policy that I created to my test mailbox, I ran the following:

PS C:\Windows\System32> Get-CasMailbox "MyTestMailbox" | set-CasMailbox -ActiveSyncMailboxPolicy "TestWP8"

Please note that:

DeviceEncryptionEnabled                  : False
RequireDeviceEncryption                  : True

 Are NOT the same thing.

"The DeviceEncryptionEnabled parameter, when set to $true, requires that the storage card of the device be encrypted. The default value is $false. We recommend that you don't use this setting and use the RequireStorageCardEncryption parameter instead."

The documentation on these parameters isn't currently very clear (imo), and varies depending on which cmdlet you enumerate, get-help get-ActiveSyncMailboxPolicy -full vs get-help New-ActiveSyncMailboxPolicy -full, so please go with the ABOVE guidance and DON'T use the DeviceEncryptionEnabled parameter. If you want the actual DEVICE to be encrypted, use the RequireDeviceEncryption parameter.

Please note that NEITHER Win 7 NOR Win 8 currently support encryption of the storage card:
https://support.microsoft.com/kb/2464593/en-us

During testing, I can see whether any applied parameters are working by running the following:

Get-ActiveSyncDevice | Get-ActiveSyncDeviceStatistics | select-object DeviceID,DeviceOS,DeviceOS,DeviceModel,DeviceAccessState,DevicePolicyApplied,DevicePolicyApplicationStatus

In the above example, you can see that the policy is listed as "AppliedInFull", and DeviceAccessState shows "Allowed".

If there are problems with any of your settings, you will see something like the following:

Note in the above snapshot that the DeviceAcessState shows "Blocked", and that the DevicePolicyApplicationStatus is "PartiallyApplied".

See https://help.outlook.com/en-us/140/cc952755.aspx for instructions on connecting to EMS via Remote Powershell.

These tests were performed against one of my Exchange Online (O365 for Education) domains. Results may vary slightly depending on YOUR version of Exchange. Enjoy!

Many thanks to Jamie Tsottles and Charlotte Raymundo for assisting with this post and verifying the veracity of the content :)

IIS 7 Primer (for those accustomed to IIS 6)

This "Primer" was condensed down from a week long IIS 7 class that I had the privilege of attending, which was delivered by Patty Stoy. You rock Patty! Laura LaFleur assisted with development of this content. Thanks Laura! And a BIG thank you to Courtney Curry for evangelizing this content and encouraging me to convert it to a blog post. Thanks Courtney :).

Props to Scott Williams for doing the technical review and proofing. Many thanks to Jason Slaughter for "The Case of the Missing G". :) And to Brian Day, for all his wonderful ideas ;).

And remember, The Wire Never Lies! - https://blogs.technet.com/b/kpapadak/

Things to get used to…IIS7 is VERY different. So different, you can forget about IIS6 for the most part. Let's Rock & Roll Peeps!

• ASP.NET and the .NET framework have been fully integrated into IIS7. This means that developers can fully extend the core server architecture by using ASP.NET and the NET framework.
• IIS7 is now very 'modular'. There are over 40 modules that 'can' be installed. However, a 'default' install of IIS7 will include only the modules necessary to provide static web content with anonymous authentication. It IS possible to have 3rd party modules as well, so watch out for these!
• IIS7 does NOT include POP3 or SMTP services by default
• To Install IIS7 via GUI using Server Manager...  For example, for a CAS server:
• Start>Run>All Programs>Administrative Tools>Server Manager
• On the left, click Roles
• On the right in the Roles frame, select Add Roles, click Next
• In the Add Roles Wizard, ensure that the following is selected: 

         Web Server

         Web ISAPI-Ext

         Web-Metabase

IIS6 Compatibility Mode

Basic Authentication

Digest Authentication

Windows Authentication

Dynamic Content Compression

HTTP Redirect (optional)

RPC Proxy (optional)

Windows Process Activation Services

Active Directory CertificateServices (optional)

• EVERYTHING that can be installed via GUI can beinstalled via the command line tool:  servermanagercmd
  
  • Try this from a CMD line:  servermanagercmd -q
  • Details on how to install prerequisites for Exchange using the command line tool, servermanagercmd:

https://technet.microsoft.com/en-us/library/bb691354(EXCHG.80).aspx

Note:  It IS important that the prerequisites are installed in the proper order (for Exchange Server) when installing via command line!

• IIS services are dependent on the Windows Process Activation Service (WAS)!  All features grouped under the IIS-Web server update depend on the WAS features so
  WAS must be installed when they are selected. 
• The GUI will automatically install WAS, but when installing by CMD or script then WAS must be explicitly specified.
  • If you MUST troubleshoot by uninstalling IIS, you MUST remove the Windows Process Activation Service as well as IIS components to get a 'true' uninstall. 
  • IIS7 install logs are located in %systemroot%\Windows\iis7.log

• By default, when you create a new site in IIS, IIS will automatically create a new application pool with the same name
• Windows Process Activation Service (WAS) will create a special <AppPoolName>.config file for the worker processes to use and will be located in the \inetpub\temp\AppPools directory. WAS will ACL
  it to only allow that <AppPool> SID to access it. This means the app pool is isolated from all other app pools!
• And, you should ALL know how SUPER COOL that is :)

• IIS6 had one long, flat file, metabase.xml, known as the “IIS Metabase”, which contained ALL settings related to IIS, all web sites, vdirs, etc.
• IIS7 no longer uses this single, long flat file. It was replaced by ApplicationHost.config. 
• Some lower level settings are delegated to web.config files.  These are hierarchical files, all written in XML, that are easily viewable/editable in any text editor, e.g., Notepad.

The 3 config files that make up the
basis of an IIS7 install are:

1. The root config file: ApplicationHost.config

• Contains definitions of all sites, applications, vDir's, app pools, as well as global defaults for the web server.
• This will generally be the ONLY config file present after a default install of IIS7 (Exchange is an exception, of course)
• This file is located in %windir%\system32\inetsrv\config

THE Two root .NET framework config files:

1. Machine.config
2. Web.config 

• Both of the .NET Framework files are located in
  %windir%\Microsoft.NET\Framework\framework_version\CONFIG.
• In addition to the root applicationhost.config file, there can be configuration files at any hierarchical level beneath it,
  including site, application, virtual, and/or physical directories. These are called web.config files, and they store settings that are particular to the specific
  level (site, virtual directory, etc) where they are stored.
• Sites, applications, virtual directories, physical directories, and files, inherit configuration settings from the server-level
  configuration files and from any parent configuration files. This means you 'could' have several configuration files that apply to any particular virtual directory.

Reference: https://technet.microsoft.com/en-us/library/cc754617.aspx

• Exchange 2007 requires a metabase, so you will be required to install II6 Compatibility Mode when installing Exchange.
  Therefore, you will get a metabase.xml file.
• web.config files are NOT machine specific! This means if you have an issue with the /owa vDir on Windows 2008 Exchange 2007/2010 server, you can copy your web.config from a "healthy" machine, to the "unhealthy" machine. COOL RIGHT! You can also do comparisons of "good" to "bad" using standard text comparison utilties. (windiff, etc)
• If you make a "bad" edit to a web.config file, you'll get the dreaded 500.19 - error indicating an error/invalid reference in a config file. Pretty self explanatory!

The GUI will 'show' you the lowest level file from which it is getting configuration settings. Click on a configuration setting within the site or directory to get this information to appear in the GUI, e.g., in IIS Manager, in the Connections pane on the left expand the DWS and select OWA. In the Features View pane on theright, double-click Authentication. Now at the bottom of the screen on the left, it shows:  Configuration:  ‘Default web site/owa’ web.config

 
  
  New IIS Accounts

• Previously, IIS used LOCAL accounts,
  IUSR_<MachineName> and IIS_WPG. There were drawbacks to these, as the LOCAL account and its SID were included in the metabase.xml and in ACLs, thus
  if you tried to copy the metabase or ACLs to another machine, it would not work.
• To replace the IUSR_MachineName account, we now have IUSR
• To replace IIS_WPG group, we now have the IIS_USRS group
• The IUSR account is a BUILT IN account. Since it is a built in account, IT NO LONGER
  NEEDS A PASSWORD! NO more worrying about resetting or syncing passwords!
• Since the IUSR account is built-in, the SID will be the same on ALL machines with IIS7 (S-1-5-17)
• Since there is no longer anything ‘machine specific’, config files and ACL’s can be copied from one machine to another with no issue.

• To view currently executing requests in a worker process, do the following!
• Open IIS Manager.
• In the Connections pane, select the server node in the tree.
• In Features View, double-click Worker Processes.
• Select a worker process from the grid.
• Click View Current Requests in the Actions pane.
• View the list of requests in the grid.

Reference: https://technet.microsoft.com/en-us/library/cc732518.aspx

• Failed Request Tracing
  • This is an EXCELLENT new troubleshooting resource available in IIS7. The module must be installed and tracing enabled to collect tracing, then rules must be created to define what you wish to trace.
    • Rather than write this out, instructions can be found here...

https://technet.microsoft.com/en-us/library/cc725948.aspx

Everyone please try this out!

• DETAILED ERRORS
• By default, a remote browser will get a LESS detailed error message than what will be displayed locally on the server. To CHANGE this behavior so that remote browsers get the same detailed error message, run the following from the directory: 
  %systemroot%\system32\inetsrv:
• Appcmd set config /section:httpErrors /errorMode:Detailed            (default is DetailedLocalOnly)

USEFUL TOOLS/ITEMS SUMMARY

• For the most part, FORGET ABOUT RIGHT CLICKING IN IIS7!! It will generally get you no where!
• The following details how to install prerequisites for Exchange using the command line tool, servermanagercmd

https://technet.microsoft.com/en-us/library/bb691354(EXCHG.80).aspx

• How to make/restore a backup – APPCMD, the new all purpose utility for IIS7,
  located in %systemroot%\system32\inetsrv directory

AppCmd Add Backup "BackupName"

AppCmd List Backup (to list all backups)

AppCmd Restore Backup “BackupName”

AppCmd Backups do NOT backup web.config files!

• Failed Request Tracing

https://technet.microsoft.com/en-us/library/cc725948.aspx

• To turn on DETAILED error reporting for remote browsers

Appcmd set config /section:httpErrors /errorMode:Detailed           

 If you are looking for something and cannot find it, IT PROBABLY ISN’T INSTALLED!

• The redirect module is NOT installed bydefault. 

• URLScan functionality comes included as an optional module. Be sure to watch out for this! This module includes something called File Extension Restrictions, which will give an error code of 404.7 when a restriction is triggered. Verbs can also be denied and will trigger a 404.6
• Other codes from this module…
• 404.13 – content too large
• 404.14 - URL is too large
• 404.15 – query string is too large
• Unless you have detailed logging turned on, the remote client will ONLY see the 404, not the sub status code! You will need to turn on detailed logging, browse locally, or check the IIS log files for the substatus code
• Event Viewer logs in Windows 2008 are .evtx files by default.  To view or configure IP addresses/port numbers assigned to a site…Highlight the site on the left, then click Bindings in the Actions pane on the far right. You can also view SSL certificates from this area.

• To view the Application Pools and Physical Path assigned to a vDir…
• Click the vDir (not right click), CLICK application settings in the Feature name pane, and click Basic Settings in the Actions pane

• To check or change App Pool Identity…Select application pools in the Connections pane, select the App Pool Identity , then click Advanced settings in the Actions pane. 
• The Advanced Settings window will pop up as shown below:

• To view handler mapping (script mapping) for a virtual directory, select a virtual directory, double click handler mappings in the middle screen. Below are settings for the RpcProxy mapping.  
• Note the ‘request restrictions’ box, which tells you what verbs are allowed, as well as what level of access is required by the handler

1.Open Exchange Management Shell on the Organizer’s mailbox server. - you can use Start > Run to do this btw (by the way)

a. The RUN box can be opened via Windows keyboard shortcuts as well - https://support.microsoft.com/kb/126449

b. And it looks like this...

2.Launch EXTRA

TIP: You will get a pop up warning that states something to the effect of "only do tracing when directed by MS Support, etc etc" Click through the warning.

3.In the Exchange Troubleshooting Assistant that launches, select the task “Trace Control”

4.In the Configuring Trace File window, select the appropriate disk volume and folder, provide a file name with a .ETL extension, and change the file size to at least 200mb then select “Set manual trace tags”

5.To the Set Trace Tags Manually window:

a.Select all 8 “Trace Types”

b.Components to Trace select:

i.Data.Storage – “ICal, MeetingMessage, Recurrence, UserConfiguration, WorkHours”

ii.InfoWorker.Assistants.Calendar – “Select All Tags”

iii.MapiNet – “MapiMessage, MapiNotification, MapiSynchroniser, tagDelegatedAuth”

iv.MSExchangeWebServices – “CalendarAlgorithm, CalendarCall, CalendarData”

v.PtTrace – “Select All Tags”

vi.Store – “tagCalendarChange, tagCalendarDelete, tagMtgMessageChange,tagMtgMessageDelete”

vii.StoreDriver – “CalendarProcessing”

c.Expand “Show mailbox trace filtering options”

i.Check the box UserDN

ii.Fill in Alias in available text box and select “Resolve Mailbox”

d.Scroll down and select Start Tracing

6.Repeat steps 1 – 5 for Meeting Attendee’s mailbox server to capture the Attendee’s data.

7.Create meeting from Organizer to Attendee, have Attendee accept meeting, observe meeting change one hour ahead on Organizer’s calendar.

8.Stop tracing on both mailbox servers.

While it IS preferable to have the Outlook logging running at the same time, there have been instances where this is not possible. VERBOSE LOGGING MUST BE ENABLED!

9. Enable Verbose Logging in Outlook by doing the following for Outlook 2003 and 2007:

For Outlook 2010:

10.Close Outlook (So logging will start). NO LOGS WILL BE COLLECTED UNTIL OUTLOOK IS RESTARTED!.

TIP: This means that you could check the box ahead of time, and have the user restart later :)

11.Collect data (reproduce the issue, or leave logging running until the "intermittent" issue occurs)

a.Collect ETL file from Organizer’s mailbox server

b.Collect ETL file from Attendee’s mailbox server

c.Collect all files from user’s workstation%TEMP%\OLKCALLogs\OLKCalLog<date/time stamp>.ETL

d.Include the subject, date and time of the meeting.

12. You can have the user disable Outlook logging at this point and continue working normally.

Per the following KB: Logging can also be enabled/disabled via the registry. Keep in mind that ENABLING logging WILL cause a performance impact. There is no way to gauge the LEVEL of impact, as this depends on the nature of the issue as well as the usage habits of the affected user. And "intermittent" issues are often difficult (or impossible) to reproduce in a sanitized (lab) environment.

TIP: This means that yes, you can likely enable logging via GPO/script/whatever your favorite way is of hacking up the registry (i didn't say that and don't blame me!).

From the following article - https://support.microsoft.com/kb/831053

For Outlook 2003 and Oulook 2007:

1.     On the Tools menu, click Options.

2.     On the Other tab, click Advanced Options.

Click to select the Enable mail logging (troubleshooting) check box in Outlook 2003 and the Enable logging (troubleshooting) check
box in Outlook 2007, and then click OK two times.

-or-

1.     Click Start , and then click Run .

2.     In the Open dialog box, type regedit , and then click OK .

3.     In Registry Editor, locate the following key for Outlook 2010:

HKEY_CURRENT_USER\Software\Microsoft\Office\14.0\Outlook\Options\Mail

Locate the following key for Outlook 2007:

HKEY_CURRENT_USER\Software\Microsoft\Office\12.0\Outlook\Options\Mail

Locate the following key for Outlook 2003:

HKEY_CURRENT_USER\Software\Microsoft\Office\11.0\Outlook\Options\Mail

4.     In the right pane, right-click EnableLogging , and then click Modify

5.     In the Edit DWORD Value dialog box, type 1 under Value data ,and then click OK .

6.     Quit Registry Editor.

Upload the data to the SFT workspace provided to you by your Engineer or PFE to have it analyzed!!!

This info was dug out of the bowels of VKB, and my brain. And technet. I have no idea who to thank for helping me with this knowledge, but I'll start with Sandra Lafrenia, David Lugaro, Laura LeFLeur, Chris Korff, Kary Wall (log parser studio ftw!), Will Duff :), Tim McMichael, Tom Moore, Richard Roddy, Ragesh Gopinathan, Tak Chow, Mathew Barbee, Elizabeth Szpala (My grammar checker  :)), Brad Hughes (testexchangeconnectivity.com), Adam Morris, Chad Lacy and Tam Pham, Cherry Canady, Harvey Lancey, Gerod Serafin, Chris Lineback, Steve Antonio, Konstantin Papadakis, Theo Browning (HB :), Scott Landry, Nino Bilic, Kalvin Koether, Dan Watson, Van Makriniotis (And the rest of Boston ;), Scott Oseycheck, Brian Day (HB) and Greg Mansius (Gold Star :)). Oh, and Brian Prince (SDP!). If I have forgotten anyone, just harangue me and I'll happily add your name :).

Oh. And Dave Goldman. And David Michael :)

I would thank TOM KERN, but we all know why that isn't necessary :p

Special thanks to Cherry Canady for grabbing most of the images from her lab!

hth,

/kris.

Well, prodcump may be able to help you.  Procdump is the newest thing around! You might remember adplus (eww!), well now its all about the Procdump! Procdump allows us to "dump" the contents of the memory at the time (hopefully) that the "issue" occurs.

https://technet.microsoft.com/en-us/sysinternals/dd996900.aspx - download procdump from this link

Here are some basic procdump usage note:

-ma full memory dump, always do this on 2003 as 4gb is not much and it is good to have the heap

-mp miniplus, 2007 – 2010 grabs the essential linked heap memory

-n 3 Typical hang dump to get 3 dumps - We typically want 3 dumps, so we can see what changes, or what DOESN'T change, in memory (i.e., is the same thread waiting in all 3 dumps?). A single dump file is typcially not that useful.

-accepteula self explanatory

-e create a dump file only when an UNHANDLED exception occurs. We're typically not interested in HANDLED exceptions, because those types of exceptions get HANDLED and your program will continue to run.

And last but not least, we need a place to store the dump file

So, an example of a dump of a typical Exchange store.exe hang would be something like this: 

Procdump -e –mp store.exe –n 3 –accepteula  d:\dumps\store_issue.dmp

This will set up procdump to monitor store.exe and when/if an unhandled exception occurs, it will create 3 dump files and store them in the specified location.

Here's another fine example, courtesy of Edwin Huber. This was used to catch an MSFTEFD crash

Another neat trick is to trigger the dump off of a performance counter. Remember, you MUST get the RIGHT data if you want to help your customer resolve their issue!

In this Example, we will catch 3 hang dumps dumps when version bucket usage is 2480:

Procdump –mp store.exe -p “\Database(Information store)\Version buckets allocated” 2480 –n 3 –accepteula d:\dumps\store_issue.dmp

Often times it can be beneficial to get perf data when gathering a procdump. This will allow the EE reviewing your dump (or you, if you're that awesome!) to get a better idea of WHERE in the dump to look. Usually you will want to gather threads in this type of perf. The easiest way to do this is to use ExPerfWiz: https://experfwiz.codeplex.com/

You will want to be sure to grab threads with the -threads switch, so your command might look something like this:

Here's a sample command:

.\experfwiz.ps1 -circular –maxsize 1024 –StoreExtendedOn –EseExtendedOn -threads -interval 30 -filepath D:\Logs

To STOP the perfwiz, run .\experwiz.ps1 -stop. The files will be of type .blg

Zip the files and upload them to the workspace provided to you.

NOTE: you might need to adjust which switches you use based on your particular issue!!

ExPerfWiz can be downloaded from https://experfwiz.codeplex.com

Once you have gathered your dump files, you will need to submit them to CTS for review. If possible, please make sure the dumps are on a share LOCAL to the office where the EE reviewing is located. So, if the EE is in Charlotte, NC, and you have a desk/cube, etc, in that location, pull the files down locally and share them out (unzipped).

For IIS issues, or anything that involves w3wp or application pool crashes, procdump may not be your best bet. Below are detailed posts on using debugdiag to generate dump files in such circumstances:

https://blogs.msdn.com/b/kaushal/archive/2012/05/09/using-debugdiag-to-capture-a-dump-on-first-chance-exception.aspx

https://blogs.msdn.com/b/pfedev/archive/2012/09/30/troubleshooting-event-id-5010-iis-application-pool-availability.aspx

Many thanks to Mark Russinovich, Mike Lagase, Edwin Huber, Theo Browning, Konstantin Papadakis and Jamie Tsottles for assistance with this blog post. And thank you to Jeff Stokes for making the suggestion :)

These were domain joined machines, which means we would be contacting Active Directory to get our Autodiscover url. Since this problem was only affecting a small subset of machines, it was assumed that autodiscover was properly configured in the environment. So, from an affected client we hovered over the Outlook icon in the bottom right corner of the machine, held down the ctrl key and right clicked, and ran Test E-mail Autoconfiguration. Since this was a domain joined client there was no need to enter credentials, so we just unchecked the "Use Guessmant and "Secure Guessmart" boxes, and hit the Test button (there should be no need to change the default populated E-mail address OR enter a password on a domain joined machine).

During the test, Outlook contacted the SCP like it should, but then returned an error I had never seen before...0x80072ee4.

Looking this error up, I found the following information...

 ERROR_INTERNET_INTERNAL_ERROR                                  inetmsg.h
 ERROR_WINHTTP_INTERNAL_ERROR                                   winhttp.h
 ERROR_INTERNET_INTERNAL_ERROR                                  wininet.h

So, now I felt like we were getting somewhere, but I still wasn't quite sure yet "where" this was :)

Sometimes proxy configurations can interfere with Autodiscover communication, so we entered the autodiscover SCP value (in the form of https://exchangeservername.domain.com/autodiscover/autodiscover.xml) into the client's web browser, and we got the expected response (an xml page with a 600 error). I felt like this ruled out any possible proxy configuration issues (even though we excluded the proxy in other ways).

In the past I have seen non Microsoft programs cause this type of problem, so we decided to "clean boot" the machine as described here: https://support.microsoft.com/kb/331796

We also disabled all non-Microsoft Outlook add-ins as described here: https://office.microsoft.com/en-us/outlook-help/enable-or-disable-add-ins-in-office-programs-HA010034127.aspx

We then rebooted the machine but were still out of luck :(

Feeling like I was running out of options, I decided to do some netsh tracing, as described here: https://technet.microsoft.com/en-us/library/dd878517(v=WS.10).aspx, using the "internetclient" scenario. Upon reviewing the trace, I found that none of the traffic seemed to be getting past winsock.

As a last ditch effort, I had the client reset winsock by running the following from an administrative command prompt:

netsh winsock reset

This command requires a reboot, and after rebooting we found that Outlook was now able to successfully contact Autodiscover (which also fixed our oof and availability issues). It turns out that a program that had been installed on the affected machines had modified Winsock in such a way as to break Outlook communication over SSL back to Exchange.

Hope this information helps you out if you ever happen to run into this error :)