Small Business Server Standard 2011

Small Business Server Essentials 2011

Migration can be classified as :

1. You want to introduce the standard / enterprise products in your current domain and remove SBS.

2. You have a new domain (already set up) and you want to move users and computers, essentially using the Active Directory Migration Tool (ADMT).

I will address these two scenarios in the upcoming blogs. There can be a number of permutations and combinations, once you are heading towards a standard environment but the basic principle remains the same.

There are some thumb-rules to any migration involving a Small Business Server:

1. You cannot take off FSMO roles and Global Catalog from SBS for more than 7 days.

2. Once an SBS Server is demoted, you cannot use it as a member server. If you wish to use the hardware, you must reinstall the Operating System to a standard server product.

3. You can have as many domain controllers as you want in an SBS Domain as long as SBS has FSMO and GC roles.

In my next blog, I will talk about migration from Small Business Server to a Windows 2008 Server with Exchange, while retaining the current domain name.

After several Service Packs, updates (both MS and Non-MS), were applied on the server, we came across this issue of not being able to start the several Exchange Services.

Events Logged were as follows:

Process MAD.EXE (PID=4036). Topology discovery failed, error 0x80040952 (LDAP_LOCAL_ERROR (Client-side internal error or bad LDAP message)). Look up the Lightweight Directory Access Protocol (LDAP) error code specified in the event description. To do this, use Microsoft Knowledge Base article 218185, "Microsoft LDAP Error Codes."

 Process w3wp.exe () (PID=3160). An remote procedure call (RPC) request to the Microsoft Exchange Active Directory Topology service failed with error 1753 (Error 0x6d9 (There are no more endpoints available from the endpoint mapper) from HrGetServersForRole). Make sure that the Remote Procedure Call (RPC) service is running. In addition, make sure that the network ports that are used by RPC are not blocked by a firewall.

 Process MSEXCHANGEADTOPOLOGY (PID=9608). When updating security for a remote procedure call (RPC) access for the Microsoft Exchange Active Directory Topology service, Exchange could not retrieve the security descriptor for Exchange server object DIAMONDSBS - Error code=80040a01.

The Microsoft Exchange Active Directory Topology service will continue starting with limited permissions.

 After several hours of troubleshooting  when we tried to reinstall Exchange, it failed with :

Error:

The following error was generated when "$error.Clear();

 initialize-ServerPermissions -DomainController $RoleDomainController

 " was run: "Could not find a directory entry for the local host.".

 Incidently it turned out to be an issue with the host file; It had the following entry:

# ::1 localhost

127.0.0.1 localhost127.0.0.1 localhost

Once we removed 127.0.0.1 localhost127.0.0.1 localhost from the hostfile, we could complete the installation and start the Services. 

We later found out that a non-MS software might've caused these changes in the hostfile.

** Just to add : This was a rare case. Usually the above errors can be caused by errors in DNS / missing SPN registrations and or missing Audit permissions.

This script is used in when it is required to reinstall Exchange 2010 server on an SBS 2011 server.

Exchange Server 2010 incorrectly changes to evaluation mode after you reinstall or repair Exchange Server 2010 in Windows SBS 2011 Standard

https://support.microsoft.com/default.aspx?scid=kb;en-US;2527626

Please refer the following links for more information:

https://blogs.technet.com/b/joscon/archive/2011/03/09/error-0xc0000034-during-service-pack-1-installations-for-windows-7-and-windows-2008-r2.aspx

https://windows.microsoft.com/en-US/windows7/why-am-i-receiving-message-system-components-missing

The objective is not to offer the ultimate solution, but to provide pointers to help get to a faster resolution.

Page cannot be displayed. (DNS Error):

A Page cannot be displayed error when browsing https://companyweb occurs because of one of the following reasons:

1. DNS record for companyweb is missing in DNS.
2. SBS Server is pointing not pointing to itself for DNS. (Usually pointing to an external DNS server which cannot resolve https://companyweb).
3. IIS is stopped or another process has taken over port 80. ( I have seen a few cases where Apache had taken over the HTTP port.)

Browsing companyweb opens the Default Web Site:

This is another common issue which could occur due to the following reasons:

1. Host header is missing from the Companyweb website (SBS SharePoint site in case of SBS 2008 and SBS 2011).
2. Companyweb website is stopped.

HTTP 403 Forbidden Error when browsing Companyweb:

This usually means that Companyweb website is not extended. If the companyweb virtual server is present in the configuration database, you simply need to re-extend the site and map it to the existing virtual server.

Cannot connect to Configuration Database when browsing Companyweb:

The error is quite self explainatory and means that SharePoint has trouble connecting to the Configuration Database. Do keep in mind that SharePoint needs to connect to the configuration database first in order to locate the content database and display the webpage.
The way to go about resolving this error is to follow the same path:

1. Ensure that instance hosting Configuration database is started. (Please refer to the previous post for default SQL instances.) *
2. The Configuration Database is connected to the instance.
3. The Database files are present at the expected location.
4. A database of higher version is connected to the instance. (Example: Connecting a SQL 2005 database to MSDE)
5. If one of the sites is giving "Cannot connect to configuration database" and others are not, you must look closely at the settings of that particular site. Most common configuration errors are in setting the Application pool identities, where the account configured as the application pool identity does not have permissions on the SharePoint database.
6. Another common cause is when a SharePoint update has failed. So the SharePoint binaries were upgraded but the configuration database wasn't. Fixing this error involves looking at the upgrade logs, resolving the errors during the upgrade and then re-run the SharePoint Products and Technologies Wizard. **

* To determine which instance is hosting the Configuration database, use the DSN registry key.
  WSS 2.0 : HKLM\Software\Microsoft\Shared Tools\Secure\ConfigDb
  WSS 3.0 : HKLM\Software\Microsoft\Shared Tools\12.0\Secure\ConfigDB
  SharePoint Foundation : HKLM\Software\Microsoft\Shared Tools\14.0\Secure\ConfigDB

** To determine the version of the Configuration Database use the Versions table.

Cannot Connect to Database when browsing Companyweb:

(Different from Cannot connect to configuration database.)

This usually means that the we were able to connect to the configuration database, query the location of the content database but were not able to connect to the content database. The possible reasons being:
1. The content database is not attached to the SQL instance.
2. The database files are missing.
3. The identity used to access database does not have login permissions to the database.

HTTP 404 The Webpage cannot be found error when browsing companyweb:

Generically speaking, the process of browsing to a SharePoint site is as follows:

A 404 error means that we are not able to locate the default document (in most cases default.aspx) in the content database. This document is created when a site collection is created in the web application.

So the probable reasons could be:

1. A site collection has not been created.
2. Recently we had several issues where we saw this issue occuring due to a failed upgrade. In this case the patch went past upgrading the configuration database, however it failed before/ at upgrading the content database in question. We then went back to determining why the upgrade failed, fixed the problems based on the upgrade log and re-ran the configuration wizard.

But a lot has already been said and written about the capabilities and possibilities of SharePoint Technologies. What I really want to focus on is how SharePoint Technologies is implemented in the Small Business Server suite.

SharePoint Technologies was first introduced to us SBS users as Windows SharePoint Services 2.0 in Small Business Server 2003. We’ve made quite a transition from there to the latest SharePoint Foundation 2010 offered in SBS 2011.

One of the reasons why SharePoint technologies have always amused me is how beautifully it integrates Internet Information Services and SQL.

Broadly speaking, this is how I see SharePoint Technologies:             

I like to isolate a SharePoint issue using the above diagram into an issue with one (or more) component(s). Once the problem component is identified, the troubleshooting thereafter is more about fixing the broken component and putting the links between the adjacent components back.

To understand how SharePoint looks from each component’s perspective, I use the chart below:

In my next few posts I would like to talk about how different it is to resolve SharePoint issues in an SBS installation as compared to a standard deployment.

I would use this post as a reference for all the troubleshooting and disaster recovery scenarios. 

I happened to work on, what could probably be my longest running Exchange installation case involving the very interesting dw20sharedamd64.msi and was fortunate (!) enough to encounter all possible flavors of this error message.

The issue started when we attempted to patch the server with Exchange 2007 Service Pack 3 which failed. When I started working, Exchange was already uninstalled and every attempt of installing Exchange was failing with the following error:

ErrorRecord: Microsoft.Exchange.Configuration.Tasks.TaskException: Installing product C:\Program Files\Microsoft\Exchange Server\bin\dw20sharedamd64.msi failed. This action is only valid for products that are currently installed. Error code is 1605. ---> System.ComponentModel.Win32Exception: This action is only valid for products that are currently installed

DW20SharedAMD64.MSI is nothing but Microsoft Application Error Reporting or Dr. Watson, better known by the installer as 000021599B0090400100000000F01FEC.It was clearly pointing us to a broken installation of Microsoft Application Error Reporting.

When installer logs (voicewarmup!) did not help much, we went back to basics and collected a Process Monitor log when trying to install the MSI.

1:55:35.0521838 PM msiexec.exe 6208 RegOpenKey HKCR\Installer\Products\000021599B0090400100000000F01FEC NAME NOT FOUND Desired Access: Read

We could get this key from another machine, but in this case, I was able to find another copy of the key at the following location:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UserData\S-1-5-18\Products\000021599B0090400100000000F01FEC

Its a simple trick of exporting this key and replacing HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion with HKEY_CLASSES_ROOT and removing UserData\S-1-5-18\.
In any case, point is to get that key back.

Once we got HKCR\Installer\Products\000021599B0090400100000000F01FEC back, we got the following error while installing:

Installing product C:\Program Files\Microsoft\Exchange Server\bin\dw20sharedamd64.msi failed. Another version of this product is already installed. Installation of this version cannot continue. To configure or remove the existing version of this product, Use add/remove programs on control panel

The error was much clearer this time around. Using Installer Cleanup we were able to find a product version 12.0.6015.5000. (I could use Add/Remove Programs as well.)
Once the product was removed we went past "Another version of this product is already installed" error, only to get the third error:

Installing product C:\Program Files\Microsoft\bin\dw20sharedamd64.msi failed. Fatal error during installation. Error code is 1603. Last error reported by the MSI package is 'Could not open key: UNKNOWN\Components\7ABFE44842C12B390AF18C3B9B1A1EE8\000021599B0090400100000000F01FEC.   Verify that you have sufficient access to that key, or contact your support personnel.'.
Fatal error during installation

This is as simple as it can get! Adding permissions to the registry key under scanner and we were able to install Exchange and subsequent service packs just fine! 

With changed patching and servicing of Exchange component we have seen issues that are very different from those experienced in the earlier versions.

The most common errors that I have seen are:

Error I: Process execution failed with exit code -1.

and

Error II: Fatal error during installation. Error code is 1603:

Based on what is logged in the Exchange Setup log the approach to resolving the installation issues would change.

Below is a list of few Exchange Service Pack installation errors, corresponding logs and what fixed the issue:

Error I: Process execution failed with exit code -1.

This error is usually related to underlying .NET problems. We need to look at the ExchangeSetup.log file closely to determine the exact issue;

Sample ExchangeSetup.log I:

Error compiling C:\Program Files\Microsoft\Exchange Server\bin\edgetransport.exe: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))

Finished compiling assembly C:\Program Files\Microsoft\Exchange Server\bin\edgetransport.exe ...

Take a process monitor trace when the setup is configuring the Hub Transport role;

This error is due to missing permissions on the following folder:

C:\Windows\assembly\NativeImages_v2.0.50727_64

C:\Windows\assembly is a special folder and in order to modify the permissions of that folder we need to use icacls;

C:\Windows\assembly\NativeImages_v2.0.50727_64 inherits permissions from C:\Windows\assembly

If the permissions are missing, you can reset them using the following command:

icacls C:\Windows\assembly\NativeImages_v2.0.50727_64 /reset

Sample Exchange Setup Log II:

Installing assembly C:\Program Files\Microsoft\Exchange Server\bin\Microsoft.Exchange.Management.SnapIn.Esm.dll

Failed to find dependencies of image C:\Program Files\Microsoft\Exchange Server\bin\Microsoft.Exchange.Management.SnapIn.Esm.dll because of the following error: Strong name validation failed. (Exception from HRESULT: 0x8013141A)

Download .NET SDK and verify/ skip the DLL in question using sn.exe.

It usually is an issue with a bad image/ downloads, so before skipping the dll in question, it is always advisable to download another copy of the service pack.

Sample Exchange Setup Log III:

C:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727\ngen.exe arguments: install "E:\Program Files\Microsoft\Exchange Server\bin\microsoft.Exchange.Rpc.dll" /verbose

[7/16/2010 12:58:20 AM] [2] Process standard output: Microsoft (R) CLR Native Image Generator - Version 2.0.50727.3053

Copyright (c) Microsoft Corporation.  All rights reserved.

Installing assembly E:\Program Files\Microsoft\Exchange Server\bin\microsoft.Exchange.Rpc.dll

Operation aborted (Exception from HRESULT: 0x80004004 (E_ABORT))

In order to resolve this issue, you must look closely at the event log for an event generated by ASP.NET 2.0.50727.0. This event is generated right after the setup failure event generated by MSExchangeSetup.

Event information is similar to: Exception information:

Exception type: FileNotFoundException

Exception message: Could not load file or assembly 'Microsoft.Exchange.Diagnostics, Version=8.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.

This is similar to the issue documented in KB: 951183: Error message when you try to uninstall Exchange Server 2007: "Process execution failed with exit code 5", where the DiagnosticCmdletController.dll file is unexpectedly unregistered.

You must search for the file in the local hard drive and re-register it, before retrying the setup.

Error II: Fatal error during installation. Error code is 1603:

This is a generic error and based on what is logged in the Exchange Setup logs, may have different resolutions.

Sample Exchange Setup Log I:

 [09/01/2011 16:41:09] [1] Installing MSI package 'C:\USER\Utils\Microsoft\Exchange\SP2\exchangeserver.msi'.

[09/01/2011 16:41:09] [1] Patch file 'C:\USER\Utils\Microsoft\Exchange\SP2\Updates\Exchange2007-KB981383-x64-EN.msp' was found in the patch directory.

[09/01/2011 16:41:09] [1] Found a valid order for the available patches.

[09/01/2011 16:41:09] [1] Patch 'C:\USER\Utils\Microsoft\Exchange\SP2\Updates\Exchange2007-KB981383-x64-EN.msp' has order '0'. The status code returned was '0'.

[09/01/2011 16:41:09] [1] Patch 'C:\USER\Utils\Microsoft\Exchange\SP2\Updates\Exchange2007-KB981383-x64-EN.msp' is valid for the current .msi package. The patch order is '0'.

[09/01/2011 16:41:09] [1] The patch attribute will be set to 'C:\USER\Utils\Microsoft\Exchange\SP2\Updates\Exchange2007-KB981383-x64-EN.msp;' for the .msi package.

[09/01/2011 16:41:50] [1] [ERROR] Unexpected Error

Notice: [09/01/2011 16:41:09] [1] Patch file 'C:\USER\Utils\Microsoft\Exchange\SP2\Updates\Exchange2007-KB981383-x64-EN.msp' was found in the patch directory.

This means that there is a post service pack rollup in the Updates directory. In this case it was 981383 which is nothing but RU4 for Exchange 2007 SP2.

Note the slip streaming rollups in the service pack is not supported.

You must remove any post service pack rollups from the updates folder.

Sample Exchange Setup Log II:

[25/03/2010 17:16:29] [1] [ERROR] Installing product C:\Program Files\Microsoft\Exchange Server\bin\dw20sharedamd64.msi failed. Fatal error during installation. Error code is 1603. Last error reported by the MSI package is 'The Windows Installer Service could not be accessed. This can occur if the Windows Installer is not correctly installed. Contact your support personnel for assistance.'.

[25/03/2010 17:16:29] [1] [ERROR] Fatal error during installation

Residual entries in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSIServer.

In one of the cases it was HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSIServer\WOW64  is left behind from a Service pack 2 installation.

Sample Exchange Setup Log III:

[8/12/2010 2:59:38 AM] [1] [ERROR] Unable to remove product with code 24b2c164-de66-44fe-b468-a46d9d5e6b31. Fatal error during installation. Error code is 1603. Last error reported by the .msi package is 'Could not open key: UNKNOWN\Components\C4DF6B15155640110AF18C3B9B1A1EE8\461C2B4266EDEF444B864AD6D9E5B613.   Verify that you have sufficient access to that key, or contact your support personnel. '.

Missing registry permissions on:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UserData\S-1-5-18\Components\<GUID mention in the error log>.

P.S.  I will continue adding more errors and solutions to this post...

This is one such peculiar scenario where some of the POP3 users, instead of connecting to the Exchange server,  download their emails separately from the ISP using their own mail client.

The trouble is when an Exchange user tries to send emails to these external users. 

** Note that these users have no Active Directory accounts and most importantly no mailboxes in the SBS Domain.

If represented graphically, the scenario would appear very much similar to the one drawn below:

When User1 tries to send mails to User2, he/she receives and NDR:

Your message did not reach some or all of the intended recipients.

The e-mail account does not exist at the organization this message was sent to. Check the e-mail address, or contact the recipient directly to find out the correct address.

<FQDN of the SBS Server #5.1.1>

Based on the Exchange version running, the solution would differ:

For Exchange 2003 (SBS 2003):

E-mail to external recipients with the same e-mail domain causes NDR messages when using the POP3 Connector.

For Exchange 2007 (SBS 2008):

E-mail to external recipients with the same e-mail domain causes NDR messages when using the POP3 Connector on Microsoft Windows Small Business Server 2008.

For SBS 2011 installations running Exchange 2010, the settings are similar to those in Exchange 2007.