Setup of the Windows Essential Business Server Management server fails while installing Microsoft System Center Essentials 2007 SP 1 (SCE). This can occur whether you are performing a new installation, joining an existing domain or a replacement mode setup.

The Windows Server Update Services 3.0 SP1 package that is installed as part of SCE 2007 SP1 is out of date. As a result it prevents the installation of SCE, and therefore EBS, from completing.  You can cross check messages in the log files in the More Information section below to determine if you are running into this.

Resolution:

Use the following steps to substitute WSUS 3.0 SP 2 for WSUS 3.0 SP 1:

1. Begin the installation of the EBS Management server and stop once you reach the EBS setup (OOBE) wizard.
2. Press Shift+F10 to open a command shell. From here, install the EBS Preparation and Planning tools, but *DO NOT launch them* after they are installed.
3. Replace WsusSetup.exe with the SP2 version using the following steps.
   1. Download the WSUS 3 SP2 x64 version. You can get it here.
   2. Rename the downloaded package from WSUS30-KB972455-x64.exe to WSUSSetup.exe
   3. Rename C:\Program Files\Windows Essential Business Server\Bin\SCE\Setup\AMD64\WSUSSetup.exe to C:\Program Files\Windows Essential Business Server\Bin\SCE\Setup\AMD64\WSUSSetup.old
   4. Copy the new WSUS package to C:\Program Files\Windows Essential Business Server\Bin\SCE\Setup\AMD64\
4. Replace the WSUS_HOTFIX with an update for the Preparation and Planning wizard using the following steps.
   1. Rename C:\Program Files\Windows Essential Business Server\bin\WSUS_HOTFIX\KB954960_x64.msp to KB954960_x64.old.
   2. Download the KB 977195 update for the Preparation and Planning wizard to the server. Shortcut here.
   3. Extract the MSP from the CAB and rename it to KB954960_x64.msp. Copy this KB954960_x64.msp file into the folder C:\Program Files\Windows Essential Business Server\bin\WSUS_HOTFIX\ .
5. Close the command shell and continue with the EBS Management setup. Select NO to updates.

More Information:

To verify that you are encountering this issue, look for the highlighted messages in the log files indicated below.

C:\Program Files\Windows Essential Business Server\Logs\MMSSetup.log

[3260],"2011/01/18 08:26:47.332","MMSSetup_InstallSCE","Error","Microsoft.WindowsServerSolutions.Setup.SceTasks.SceInstallTaskException: The System Center Essentials installation failed with exit code 0xFFFFFFFF."

[3260],"2011/01/18 08:26:47.332","MMSSetup_InstallSCE","Verbose","Exit: InstallSCE.Install: Void Install()"

[3260],"2011/01/18 08:26:47.442","Task","Verbose","Exception = Microsoft.WindowsServerSolutions.Setup.SceTasks.SceInstallTaskException: The System Center Essentials installation failed with exit code 0xFFFFFFFF.

at Microsoft.WindowsServerSolutions.Setup.SceTasks.InstallSCE.Install()
at Microsoft.MMS.Setup.Deployment.Core.Management.InstallSCETask.Configure()
at Microsoft.MMS.Setup.Deployment.Core.SetupTask.RunInstall(ITaskDataLink tdLink)
at Microsoft.MMS.Setup.Deployment.Core.SetupTask.Run(ITaskDataLink tdLink)"

[3260],"2011/01/18 08:26:47.442","Task","Verbose","Level 1: Exception type = Microsoft.WindowsServerSolutions.Setup.SceTasks.SceInstallTaskException"

[3260],"2011/01/18 08:26:47.442","Task","Verbose","Level 1: Exception message = The System Center Essentials installation failed with exit code 0xFFFFFFFF."

[3260],"2011/01/18 08:26:47.442","Task","Verbose","Level 1: Exception Data:"

[3260],"2011/01/18 08:26:47.442","Task","Verbose","thrown exception type = Microsoft.WindowsServerSolutions.Setup.SceTasks.SceInstallTaskException, server role Management"

[3260],"2011/01/18 08:26:47.442","Task","Verbose","No ThrowsAttribute exception match found, and unknown exception type Microsoft.WindowsServerSolutions.Setup.SceTasks.SceInstallTaskException"

[3260],"2011/01/18 08:26:47.442","Task","Verbose","attribute exceptiontype = Microsoft.WindowsServerSolutions.Setup.SceTasks.SceInstallTaskException"

[3260],"2011/01/18 08:26:47.442","Task","Verbose","Found error message = System Center Essentials installation failed."

C:\Program Files\Windows Essential Business Server\Logs\SCESetup0.Log

8:26 AM: Drive C:\Program Files\System Center Essentials 2007 is valid

8:26 AM: SetupData::ValidateAllInput passed

8:26 AM: LaunchWSUSPrereq: Launching C:\Program Files\Windows Essential Business Server\Bin\SCE\Setup\AMD64\WSUSSetup.exe with arguments: /Q /P PREREQ_CHECK_LOG="C:\Users\ADMINI~1\AppData\Local\Temp\WSUSPrereqReport.xml"

8:26 AM: LaunchWSUSPrereq: We did not pass the Prereq check. Return Value was 5216

8:26 AM: WSUS Prereq failed!

--------------------------------------------------------------------------------

Finished Logging for Setup.exe 8:26:47 AM Tuesday, January 18, 2011

C:\Program Files\Windows Essential Business Server\Logs\WSUSSetup.log

2011-01-18 08:26:47 Success MWUSSetup Validating pre-requisites...

2011-01-18 08:26:47 Error MWUSSetup Failed to determine if an higher version of WSUS is installed. Assuming it is not... (Error 0x80070002: The system cannot find the file specified.)

2011-01-18 08:26:47 Error MWUSSetup WSUS is outdated. But this will not block setup (Error 0x00000000: The operation completed successfully.)

2011-01-18 08:26:47 Success MWUSSetup Pre-requisites file: C:\Users\ADMINI~1\AppData\Local\Temp\WSUSPrereqReport.xml

2011-01-18 08:26:47 Error MWUSSetup Some prerequisite check failed. Look at C:\Users\ADMINI~1\AppData\Local\Temp\WSUSPrereqReport.xml for details. (Error 0x80041460)

2011-01-18 08:26:47 Error MWUSSetup DoInstall: Wsus setup failed (Error 0x80041460)

C:\Users\ADMINI~1\AppData\Local\Temp\WSUSPrereqReport.xml

<Components>
<Component>Microsoft Windows Server Update Services 3.0</Component>
</Components>
<Description>You cannot install WSUS with a higher version of WSUS already installed.</Description>
<Resolution>For information regarding how to resolve this issue, go to https://go.microsoft.com/fwlink/?LinkId=71220</Resolution>
</Result>
<Result StatusCode="2">
<Components>
<Component>Microsoft Windows Server Update Services 3.0</Component>
</Components>
<Description>Current WSUS version is out of date.</Description>
<Resolution>For information regarding how to resolve this issue, go to https://go.microsoft.com/fwlink/?LinkId=74472</Resolution>
</Result>

The EBS Migration Kit includes 3 licenses of Windows Server 2008 Standard, 2 licenses of Microsoft Exchange Server 2007 Standard, and Microsoft System Center Essentials 2007 (with 10 Server Management licenses and 50 Client Management licenses). The kit is available for a limited time, from June 30^th, 2010 through December 31^st, 2010. Following December 31^st, the EBS Migration Kit will no longer be offered and customers who wish to move to the individual products will need to purchase them through traditional channels. Customers who acquire the EBS Migration Kit today do not need to migrate out of the suite immediately.

Customers can acquire the EBS Migration Kit until December 31^st through a simple online order process at https://www.microsoft.com/ebs/en/us/offers.aspx.

Please visit the EBS Website to learn more about the EBS Discontinuation.

We wanted to resurface a continuing support issue with deployments that we first commented on back in December of 2009, and again in a March post on the Microsoft Enterprise Networking Team Blog. After a very encouraging decrease in support incidents traced to this issue, we’ve seen an upward trend again for the same issue – perhaps a reminder will reverse the trend!

Solution

If you are running certain older versions of Symantec Endpoint Protection or Symantec Antivirus, you can get the solution from Symantec. Symantec confirms that this is a known issue and there are updates to resolve the problem.. For more information, please see the Symantec Knowledge Base article about this issue . For information about Symantec support for Endpoint Protection 11, visit the Symantec Support site. (https://www.symantec.com/business/support/index.jsp)

Please note: If you are unable to upgrade promptly or remove the software, Symantec urges you to contact their technical support to determine if there are any workarounds available to you.

Problem Indications

While the error messages are not specific to this issue, and can often have other unrelated causes, versions of Symantec Endpoint Protection prior to version 11.0.4202 or Symantec Antivirus 10.2 on a Windows Server can cause these connectivity problems. The real frustration can be the difficulty in determining that the root cause may be due to these third-party products; the intermittent loss of productivity and reliability of client connections to the server often results in costly and time-consuming investigations. To make the matter even more frustrating, it does not happen immediately after installation of the security software, the interoperability issue can sometimes happen weeks after deployment.

• Symantec Endpoint Protection versions prior to 11.0.4202 (MR4-MP2). Older versions of Symantec Symantec Antivirus (prior to 10.2) can also cause this problem. The problem occurs when you have the Autoprotect feature enabled in the applications.
• The problem can affect 32-bit and 64-bit versions of Windows Server 2003, 2008, 2008 R2, including Windows Small Business Server 2003, Windows Small Business Server 2008, and Windows Essential Business Server 2008.
• The problem is intermittent, can affect client connectivity within hours to up to a week after installing the applications. Usually a server reboot will restore connectivity for a short period before the problem happens again. You generally are able to ping and RDP to the server when the issue occurs, but shares are inaccessible using either \\IPADDRESS or \\ServerName\Share operations.
• The problem results from a deadlock in SRTSP.SYS or SRTSP64.SYS in a push lock operation, which then causes blocked kernel mode server threads handling SMB negotiation requests. Network traces will show the server not responding to the SMB dialect packet.

Here are the most common symptoms you may experience that may help you diagnose this issue. Note that if you have this software installed, the easiest troubleshooting step to determine if this is the cause of connectivity issues is to simply remove the software and reboot the server.

• Error message: “The network path was not found” or “The specified network name is no longer available” when attempting to open shares, map a drive, run DCDIAG to the to the affected server, use netdom to reset secure channel
• Error message: “RPC Server is unavailable” when trying to connect via Active Directory Users and Computers
• Error message: “RPC Server is too busy to process the request” when attempting to join the Windows server domain
• Error message: “No network provider accepted the given network path” or “File or network path no longer exists” when copying a file over the network to affected servers
• Printing issues (cannot update printer IP address via DNS)
• AD replication failures
• Cluster service fails to start, or inability to access existing File Share resources even if they are online according to the Cluster Administrator snap-in
• Event log Event ID 4226 and or 2022 may occur frequently (up to every 20 to 30 seconds)
  

More Information

KB 961293 Unable to access Shares "The specified network name is no longer available" when Symantec Endpoint Protection prior to 11.0.4202 (MR4-MP2) or Symantec Antivirus 10.2 are installed on a Windows 2003, 2008 or 2008 R2 Server

KB 961654 A file sharing connection to a Windows Server 2008-based server drops unexpectedly if the server has Symantec Endpoint Protection prior to 11.0.4202 (MR4-MP2) or Symantec Antivirus 10.2 installed

KB 948732 Network shares become unresponsive after some time on a Windows Server 2003 or 2008 or 2008 R2-based-based computer running Symantec Endpoint Protection prior to 11.0.4202 (MR4-MP2) or Symantec Antivirus 10.2, and you receive an error message

KB 923360 You may experience various problems when you work with files over the network on a Windows Server 2003-based or Windows 2000 Server-based computer

All of the articles also refer to the Symantec Knowledge Base article about the issue, including information about how you can obtain the fix.

** This post has been re-written on 8/25/2011 to cover a wider range of scenarios that have been generating an increasing amount of traffic recently.

The EBS 2008 Messaging Server is installed with two Exchange certificates; both are used for securing communication with the Security server. Specifically, one certificate is used by the Hub Transport role to secure EdgeSync communication with the Edge Transport Role and the other is for securing web communication between Forefront TMG and the Exchange virtual directories (this same certificate is also used for any application that uses port 443, such as OWA, Outlook Anywhere, TSGateway, and Remote Web Workplace).

Certificate Expiration Issues

The certificate used for Web traffic expires two years after its creation date. When it does, you will not be able to access resources such as OWA, ActiveSync, and RWW from the internet.

The certificate used for EdgeSync is valid for only one year from its creation date and you will start receiving warnings in Event Viewer once it expires:

Application
MSExchangeTransport
Error
12014
Microsoft Exchange couldn't find a certificate that contains the domain name Domain_Name in the personal store on the local computer. Therefore, it is unable to support the STARTTLS SMTP verb for the connector Default Server with a FQDN parameter of FQDN. If the connector's FQDN is not specified, the computer's FQDN is used. Verify the connector configuration and the installed certificates to make sure that there is a certificate with a domain name for that FQDN. If this certificate exists, run Enable-ExchangeCertificate -Services SMTP to make sure that the Microsoft Exchange Transport service has access to the certificate key.

Application
MSExchangeTransport
Warning
12015
An internal transport certificate expired.

NOTE: The certificate simply expiring will not cause EdgeSync or SMTP mail flow to stop working. You will just receive the above events and STARTTLS will be unavailable.

Recreating the Certificate

The following method should be used when either certificate has expired or has been accidentally deleted. This will fix both loss of web access and loss of mail flow between the Security and Messaging servers. Essentially we are replacing both certificates with single new one that will be valid for another 2 years. It will be issued by the trusted internal Certificate Authority while maintaining both original names to preserve certificate trust between the two servers.

Create the new certificate

1. On the Messaging Server, launch Exchange PowerShell as admin.
   
2. Run the following command to create a temporary directory for this process:
   md c:\EBSBlog
   
3. Run the following commands to collect the $ComputerName, $FQDN, $Date, and $FriendlyName values you will need:
   $ComputerName = gc env:computername
   $DnsDomain = gc env:userdnsdomain
   $FQDN = $ComputerName + “.” + $DnsDomain
   $Date = Get-Date –Format d
   $FriendlyName = $Date + “ EBSBlogCert”
   
4. Run the following command to generate the new certificate request:
   New-ExchangeCertificate –GenerateRequest –Path c:\EBSBlog\NewEBSCert.req –DomainName $ComputerName, $FQDN –FriendlyName $FriendlyName
   
5. Run the following command to submit the request:
   CertReq –attrib “CertificateTemplate:WebServer” c:\EBSBlog\NewEBSCert.req
   
6. When prompted, select your internal Certificate Authority and save the resulting certificate with the name NewEBSCert.cer back to the c:\EBSBlog folder.
   
7. Run the following commands to import the certificate into Exchange and assign the necessary services:
   Import-ExchangeCertificate c:\EBSBlog\NewEBSCert.cer
   Get-ExchangeCertificate | Where {$_.FriendlyName –match $FriendlyName} | Enable-ExchangeCertificate –Services IIS,SMTP,POP,IMAP
   
8. Select Y to overwrite the existing default SMTP certificate.

If the original transport certificate had been deleted previously, it is expected that you will receive the following informational/warning message:

New-ExchangeCertificate : The internal transport certificate for the local server was corrupted or missing in Active Directory. The problem has been fixed. If you have existing Edge Subscriptions, you must subscribe all Edge Transport servers again by using the New-EdgeSubscription cmdlet in the Exchange Management Shell.

Whether or not you receive this message, follow the steps below to recreate the Edge Subscription between the Hub role on Messaging and the Edge role on Security.

Recreate the Edge Subscription

From Security Server:

1. Open Exchange PowerShell as administrator and run the following commands:
   md c:\EBSBlog
   New-EdgeSubscription –FileName c:\EBSBlog\new.xml
   
2. Choose Yes to confirm.
   
   From Messaging Server:
   
3. Copy the xml file from the Security server to the c:\EBSBlog folder on the Messaging server and run the following command:
   New-EdgeSubscription –FileName c:\EBSBlog\new.xml –Site Default-First-Site-Name
   
4. Wait between 5 and 15 minutes, then test EdgeSync by running the following command to ensure that it’s working properly:
   Test-EdgeSynchronization
   
   

Once all entries say Synchronized, mail flow should be restored. If you are still having issues, you may want to contact Microsoft Product Support Services for assistance.

More Information

Missing Certificate

We often find that customers discover the expiration on their web certificate before they realize that their EdgeSync certificate has expired, since only the web certificate expiration will cause major functionality loss. We also find that many times customers are unintentionally removing the old certificates improperly during the course of troubleshooting, which can then break both inbound and outbound mailflow as well. In this circumstance, you may receive errors such as the following:

Application
MSExchangeTransport
Warning
12023

Microsoft Exchange could not load the certificate with thumbprint of <thumbprint> from the personal store on the local computer. This certificate was configured for authentication with other Exchange servers. Mailflow to other Exchange Servers could be affected by this error…

Application
MSExchangeTransport
Error
2018

Outbound direct trust authentication failed for certificate <SubjectName>. The target IP address of the Exchange server that Microsoft Exchange tried to authenticate to is <IPAddress>. Make sure EdgeSync is running properly.

Application
MSExchange EdgeSync
Error
1032

No credential was found for <Edge Server FQDN> …

451 4.4.0 Primary target IP address responded with: “454 4.7.0 Temporary authentication failure.” Attempted failover to alternate host, but that did not succeed. Either there are no alternate hosts, or delivery failed to all alternate hosts

There is a common misunderstanding as to what a proper backup and restore procedure should be for an EBS 2008 Management or Messaging server running as a virtual machine. We see many customers who are taking “snapshots” in their chosen hypervisors and are under the assumption that they can successfully restore them in disaster recovery or migration scenarios. Another common method we see is the use of 3^rd party backup solutions that create volume images of the server and store them in a centralized network location. This post will shed light as to why methods like this are unsupported, what the supported procedures are, and how to recover from an improper restoration of the SBS server.

As you are aware, the EBS 2008 Management and Messaging servers are domain controllers in the root of the Active Directory forest. The contents of an Active Directory database and changes to them are tracked and replicated using metadata attributes called Update Sequence Numbers (USNs) and High Watermark Vectors (HWMVs). When a domain controller is restored from backup, another metadata attribute called an invocation ID is used to notify other domain controllers in the network that this server is indeed being reintroduced into the current domain from a previous state. Depending on the type of recovery you are performing, the restored domain controller’s copy of Active Directory will either absorb the up-to-date changes from its replication partners (non-authoritative) or its replication partners will absorb its copy (authoritative).

Every restoration of a domain controller must follow the above framework. The only backup type that will properly preserve critical metadata attributes mentioned above is the System State Backup. The only restore type that will utilize these attributes to properly reintroduce the server into the existing domain is the System State Restore. Any other backup/restore method, including virtual machine snapshots, volume image snapshots, and offline copy backups of the NTDS database system files will put your server in a state called USN rollback if performed. We will discuss USN rollback in more detail in a little bit.

There are definitive methods for backing up the system state data of an SBS server that are supported by Microsoft. The following strategies available to you are:

• Windows Server Backup: You can use the built-in GUI backup utility to backup your server at the per volume level. If you choose to Enable system recovery, you will backup all drives that include OS data required for system state restore. When it comes to recovering your data, you can target just the system drives, just the system state data, single applications (Sharepoint or Exchange) or the entire server including the ability to perform a bare metal restore.
• Wbamin: This application is used beneath the surface by the SBS 2008 Backup Wizard and Windows Server Backup. Available as a command line utility, it allows you to backup just the system state data in your single targeted backup job. This is always useful in scenarios where you are about to make a change to critical system data and you want to protect yourself first without having backing up the entire drive. Please see our previous post that includes the “how to” steps.
• Data Protection Manager
• Any third party backup application that takes an Active Directory-aware system state backup using Microsoft supported backup APIs or by using Microsoft’s provided Volume Shadow Copy VSS writers (NTDS writer). Check with the particular vendor to verify whether the application meets these criteria.
• Windows Server Backup on the Hyper-V host machine can be enabled to backup the entire Hyper-V infrastructure, including all guest virtual machines. The following KB article gives all of the details, however here are a couple important points and caveats:
  • You must first register the Hyper-V VSS writer with Windows Server Backup through a registry edit.
  • You cannot target individual Hyper-V components or virtual machines for backup or restore. It is an all or nothing operation.
  • Virtual machines must have Integration Services installed and the Backup (volume snapshot) service must be enabled in order to be backed up.
  • Virtual machines that contain dynamic disks will not be backed up through this method.
    
    We recommend that you use this method to supplement your regularly scheduled backups that you should be taking from within the virtual machine guest OS itself. Due to its inability to target individual virtual machines for backup and restore, we do not recommend that you use this as your primary method.

USN Rollback

When a proper system state restore is performed, the invocation ID attribute of the restored database is reset, which notifies each replication partner that this server has indeed been restored from a previous state. Replication attempts from the restored DC to its partners would include this new invocation ID along with data that it has previously replicated. As long as the partner receives the previously acknowledged USN with a new invocation ID, there are no issues.

USN rollback happens to a domain controller when it replicates previously acknowledged USN with the same invocation ID. In this circumstance, there is no way to ensure that the changes pushed from this particular DC are the latest. When this inconsistency is recognized by its partner, they will notify it. The source server will then pause its netlogon service and disable outbound replication. This is to prevent any changes originating from this server’s copy of AD from being pushed out to the domain. At this point, you must remove this server from the existing domain and promote it back in as a clean domain controller. The following KB article goes into greater detail, including the events you will receive and includes all of the recovery steps that are required.

Note: Besides restoring a DC improperly, disconnecting a domain controller from the domain beyond 180 days (tombstone lifetime) and then bringing it back into the network will result in the same scenario.

System Center Essentials 2007 (SCE) is a unified management solution that is included with the Essential Business Server package. The application is installed on the Management Server and is designed to give the network administrator the ability to proactively manage IT environments through the use of monitors and alerts. This article addresses some “Best Practices” regarding database maintenance and performance. It also outlines steps to remove unnecessary data from the SCE databases if the databases reach the 4GB capacity.

SCE gathers agent data through predefined objects, monitors and filtering rules that are installed through Management Packs. An initial set of Management Packs are installed during the SCE installation providing basic functionality, with additional Management Packs available through the Management Pack Catalog. Data is then gathered by various scripts and detection methods outlined within these Management Packs, and inserted into SCE’s databases for reference and reporting.

In order to keep track of the information gathered, SCE installs with Microsoft SQL Server 2005 Express Edition. Even though the Microsoft SQL Server 2005 Express Edition Express edition is conveniently included, it also includes a database size limitation of 4GB (log files excluded). Depending on variables like the number of users, number of Management Packs installed and the number of day’s data is kept, it is possible for the SCE database to reach its 4GB capacity within a few months time.

WSUS Server Cleanup Wizard

Before we get into database grooming, a “Best Practice” to consider when developing a server maintenance plan would be incorporating the WSUS Server Cleanup Wizard into your monthly maintenance “to-do” list. The WSUS Server Cleanup Wizard allows you to remove out-of-date update files, superseded updates and computers that are no longer available. The recommendation by the WSUS team is to “use the Cleanup Wizard on a regular basis”, which would be considered once a month if possible, or at least once a quarter. For more information concerning the WSUS Server Cleanup Wizard: https://technet.microsoft.com/en-us/library/cc708578(WS.10).aspx.

Defining Database Grooming Settings.

“Out of the box”, SCE has a set of default Database Grooming Settings that specify the amount of time (number of days), data is kept before it is removed from the database. These settings address records like Resolved Alerts, Event data and Performance data (See the screenshot below for the full list) which can rapidly fill a database.

The “Records to delete” settings, shown in the Database Grooming Settings window below, is purely an initial baseline configured during the SCE installation. These settings can be easily modified at anytime, and can be safely set to a 3 day period.

To view\edit the Database Grooming Settings:

Open the System Center Essentials console

Select administration or the   icon

Select Settings on the left and then Database Grooming on the right.

This will open the Global Management Settings window.

How do I modify the Database Grooming Settings?

Highlight the subject in question.

Select the edit button to modify.

Change the number of days (from 7 to 3 days).

Select OK.

The objects listed under “Database Grooming Settings” are a small subset of all of the objects stored in the SQL database. These settings will help with basic maintenance, but depending on the number of SCE agents returning data and the type of information SCE is gathering, simply modifying these settings may not prevent the SQL database from reaching its limit.

What happens if a System Center Essentials database does reach the 4GB limit?

One of the first things that you notice is an error displayed in the Essential Business Server console under the “Computers and Devices” tab: “There are problems displaying data on this page”. If you try to open the SCE console it will fail to open also.

Event viewer will return errors similar to:

Log Name: Application
Source: MSSQL$SCE
Event ID: 1827
Task Category: (2)
Level: Error
Keywords: Classic
Description:
CREATE DATABASE or ALTER DATABASE failed because the resulting cumulative database
size would exceed your licensed limit of 4096 MB per database.

Log Name: Application
Source: MSSQL$SCE
Event ID: 1105
Task Category: (2)
Level: Error
Keywords: Classic
Description:
Could not allocate space for object 'dbo.PerformanceData_xxx'.'idxc_PerformanceData_xxx_TimeSampled' in database 'OperationsManager' because the 'PRIMARY' filegroup is full. Create disk space by deleting unneeded files, dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup.

**Note: the error will point to the database in question. Another database that is utilized by SCE is the OperationsManagerDW database. If this database is specified in the error, then the SQL query listed below will need query the OperationsManagerDW database instead of the OperationsManager database that is used in this example below. **

What is the actual size of the Operations Manager database?

Open the SCE instance in SQL Server management Studio Express

Select New Query

USE OperationsManager (database specified in the error above)

exec sp_spaceused

Execute Button (Shown Below)

This will return the current database size.

Q & A

Q.  Can I Upgrade SQL? Can I uninstall/reinstall SCE?

Q.  If the database has reached its limit, can I upgrade to a different version SQL 2005 to remove the 4GB limitation? Can I uninstall/reinstall SCE in order to reset the database?

A.  The answer to both of these questions is “NO”.  This is not supported.

Q.  Can I Upgrade SQL 2005 Express instance to a Standard or Enterprise version of SQL?

A.   No - this is not supported with Essential Business Server.

(https://technet.microsoft.com/en-us/ebs/dd183189.aspx -

Q.  Can I manually uninstall/reinstall SCE in EBS?

A.   No – Manually uninstalling reinstall SCE is not supported with EBS.

(https://technet.microsoft.com/en-us/ebs/dd183189.aspx

Q.  Can I uninstall SCE?

A. SCE is integrated into EBS 2008 and cannot be uninstalled.

** You can remove System Center Essentials through Programs and Features; however there is not a way to reinstall it manually once it’s been removed. If System Center Essentials has been removed, a Replacement Mode install is the only option to reinstall SCE. Do not uninstall System Center Essentials!**

Q.  How can I reduce the amount of unnecessary data currently stored in the SQL databases if the databases have reached the 4GB limit?

A.  In addition to modifying the Database Grooming Setting, you can reduce the amount of data currently stored by modifying the number of days that data is retained directly through SQL.

Before making any changes to any of the SQL databases, make sure you have created a full backup of the OperationsManager, and the OperationsManagerDW before starting. (https://msdn.microsoft.com/en-us/library/ms187510.aspx).

To view the current Maximum Days data is kept

Select New Query

Use OperationsManagerDW

SELECT MaxDataAgeDays, * FROM StandardDatasetAggregation

Execute button

In this example we see several datasets that hold data up to 37 days.

(If you want to get an idea of what a particular DatasetId is responsible for, scroll to the right, under BuildAggregationStoredProcedureName or GroomStoredProcedureName and it will return the Aggregate).

How can I modify the number of days that data is kept in the database?

The following procedure changes any settings greater than 30 days to 15 days.

To change the number day’s data is kept from 30+ days to 15 days.

Select New Query

Enter the following into the blank query window

UPDATE StandardDatasetAggregation

SET MaxDataAgeDays = 15

WHERE MaxDataAgeDays > 29

Select execute

Once the changes have been made, you can verify the days have been modified by following the steps listed above (To view the current Maximum Days data is kept).

You will need to allow at least 24 hours for all of the grooming processes to complete. Once the grooming processes have completed, the database will be available and the SCE console should open without error.

Additional Resources:

https://blogs.technet.com/kevinholman/archive/2007/10/18/useful-operations-manager-2007-sql-queries.aspx

During the development of EBS v2, we tackled quite a few challenges in integrating Microsoft server technologies. We developed solutions and workarounds, proved them through exhaustive testing, and implemented them in EBSv2. Due to external factors, EBS v2 did not release to public. Therefore, we decided to gather some of our solutions and findings, and share it with our community.

1. Performing an 'isolated' installation of Exchange 2010

There is no supported way to rollback a failed Exchange installation from the forest. However, there is a proven way to control and contain the Exchange installation inside the forest, so that in case it fails and blocks, it doesn't spread across the domain and leave the forest in an undesirable state. We implemented this approach in our automated EBS v2 installation and verified it in Beta and RC0 releases. A generalized version of this solution is presented below:

1. Select the Schema master domain controller in the domain you wish to install Exchange in. You must select the Schema master because Exchange installation starts by extending the schema and this operation c6. Use n only be targeted to the schema master.

2. Ensure a full inbound and outbound replication has completed to this domain controller and the DC is healthy (no concerning errors in eventlog, dcdiag, etc.).

3. Disable the outbound replication channels to the DC. You can do this with repadmin.exe /options +DISABLE_OUTBOUND_REPL. This step effectively isolates the DC from the rest of the domain and all changes on this DC will remain exclusively on this DC.

4. Install Exchange 2010 on any server in the domain that can contact the isolated DC, while targeting the installation to the isolated DC. It is very important to target the installation to that DC, else Exchange will pick any DC that finds best. You can use Setup.cmd /DC:<isolated DC name> to target the isolated DC.

5. If Exchange installation fails and you get blocked recovering from the failure, simply force-demote the DC and re-promote it again. This demotion will take all the Exchange residues with it and the new DC, like all other DCs will be unaware of the Exchange failure.  

6. If the Exchange installation succeeds and passes your verification, then re-open the replication channels (repadmin.exe /options -DISABLE_OUTBOUND_REPL) and allow a few minutes for all DCs to see the changes. The replication latency depends on the size of the domain and speed of connections.  

2. How to work around FSE installation issues on an Exchange server running on top of a DC?

If you are installing the Forefront Security on an Exchange server the setup executable will stop the Exchange Information Store and Transport services, configure Exchange, and restart the services. FSE will wait up to 120 seconds for the Exchange services to start up again and if they don't, it will error out. Typically Exchange Information Store and Transport should both start within 120 seconds, however, we did experience that occasionally the service takes longer to start, especially when Exchange is installed on a Domain controller.

If you encounter such a problem with FSE installation, simply stop the Exchange Transport and Information Store services and retry FSE setup. FSE will only attempt to start the services if they were initially running. Else, it will simply complete setup, report success, and leave the services untouched. FSE installation typically requires a reboot, which will automatically restart the Exchange Information Store and Transport services.    

3. How we solved problems with Exchange running on a Domain Controller in EBS

The detailed article can be found at:

https://blogs.technet.com/essentialbusinessserver/archive/2009/06/25/how-we-solved-problems-with-exchange-running-on-a-domain-controller-in-ebs.aspx

4. How to recover EBS 2008 Messaging server objects after an accidental Active Directory server object deletion

The detailed article can be found at:

https://blogs.technet.com/essentialbusinessserver/archive/2010/01/27/how-to-recover-ebs-2008-messaging-server-objects-after-an-accidental-active-directory-server-object-deletion.aspx

5. Integrating the all new Exchange Pre-deployment analyzer into deployment planning

Several years ago Exchange released an Exchange BPA v2.8 that could be deployed on (almost) any domain-joined machine and examined the 'readiness' of the domain and forest for an Exchange installation. Exchange discontinued this stand-alone tool until recently, that they released Exchange Pre-deployment analyzer for Exchange 2010. This tool is also standalone and can even run from a domain-joined Vista client. You can find the ExPDA at: https://www.microsoft.com/downloads/details.aspx?FamilyID=88b304e7-9912-4cb0-8ead-7479dab1abf2&displaylang=en

We certainly recommend that as part of your deployment planning run the readiness checks of this tool. You can also automate the process with a VB script that executes ExPDACmd.exe, dumps the report in an XML file, and read the Errors out of the file using searchExpression = "//Message[@Error='Error']".

6. Use the Exchange Remote Connectivity Analyzer:

The https://www.testexchangeconnectivity.com is a great resource for anyone to test whether they have setup the ActiveSync, EWS, RPC over Http, SMTP, etc. settings correctly. The ActiveSync test will simulate a mobile device and test connectivity. The Outlook tests simulate an Outlook connecting via Outlook Anywhere. The SMTP tests will walk through all steps for an inbound and outbound email delivery.  

7. Installing Remote desktop gateway and publishing in ISA/Forefront TMG

https://technet.microsoft.com/en-us/library/cc771530(WS.10).aspx provides in depth details on how to configure Remote desktop gateway (formerly TS gateway) in various scenarios.

8. Publishing OWA in ISA

Following article provides information on how to configure OWA in ISA https://support.microsoft.com/kb/290113

Today, we announced the decision to streamline our server product portfolio and will discontinue future development of EBS, effective June 30^th, 2010.

Since the launch of EBS, several changes have occurred that drove our decision to streamline our server product portfolio. First, midsize businesses are rapidly turning to technologies such as management, virtualization and cloud computing as a means to cut costs, improve efficiency, and increase competitiveness. Those capabilities are already available through other offerings, including Windows Server 2008 R2, Microsoft System Center and the Microsoft Business Productivity Online Suite (BPOS).

Secondly, Microsoft remains fully committed to small and medium-sized businesses. We recognize that ending future editions of EBS could result in additional cost and complexity for our EBS customers. Therefore, we are going to provide a limited offer for all EBS 2008 customers. Beginning on June 30, 2010 through December 31, 2010, current EBS 2008 customers can get the individual component software from the EBS 2008 suite for free (local taxes, shipping and handling charges may apply). To learn more about this offer, EBS customers can visit https://www.microsoft.com/ebs.

This decision not to ship future versions of EBS does not come lightly and will not impact any other Windows Server products and solutions, including the next version of Windows Small Business Server (SBS).  As a matter of fact, we are working hard to build the next version of SBS and look forward to a second decade of success with this award winning small business offering.  

There is a tremendous amount of talent and expertise on the EBS product development team and Microsoft is committed to transitioning members of the EBS product team to work on other projects within the Microsoft Server and Cloud division.

Finally, in building these products for midsize customers we’ve had the unique opportunity to work closely with customers and partners worldwide. We would like to sincerely thank those of you who provided feedback, participated in the development, and used Windows Essential Business Server.

We look forward to continuing to provide world-class Microsoft solutions for the future!

More Information:

Current EBS customers can look forward to a support cycle that holds true to the Microsoft five-year mainstream and five-year extended support lifecycle. All service packs will also be supported according to lifecycle support. For more detailed information, please visit https://support.microsoft.com/lifecycle.

For customers interested in transitioning off of EBS, we will offer a six-month promotion where customers can migrate to many of the stand-alone products included in EBS 2008. For more details on the offer, please go to https://www.microsoft.com/ebs.

Customers with Software Assurance will be offered an open agreement to receive the license for many stand-alone products that would have been included in EBS 2010. Customers who do not have Software Assurance will have the option to take advantage of the offer which begins on June 30, 2010 through December 31, 2010. For more details on the offer, go to https://www.microsoft.com/ebs.

For more information about this decision, please visit https://www.microsoft.com/ebs.

https://www.surveymonkey.com/s/LHJBSJ9.

Thanks!

From Customer Support feedback, we have learned that some customers facing a disaster on one of the Essential Business Server 2008 servers will naturally attempt to clean the servers' residue from Active Directory before starting a replacement/recovery. This is not recommended by EBS and will block the recovery solution for EBS Messaging server since EBS relies on preserved computer objects in Active Directory  to proceed with replacement and recovery. Without the original computer objects, Exchange Server recovery will not succeed. Creating new computer objects with the same name as the original EBS servers is not an option and the new computer object cannot be used for replacement.

More specifically, we have seen customers who encounter disaster clean up the residue server objects metadata using ntdsutil. Meta data cleanup will remove the ntds settings object, all replication links, server object, etc. from Active Directory. After that, they often proceed to remove the computer object from the domain partition and start replacement in what they believe is a "clean" environment. Currently, our setup expects both the computer object and the ntdsSettings object to be present in Active Directory before it proceeds with recovery, else Exchange recovery during will fail. We have included documentation to prevent users from doing this, but this seems to still be a natural and common mistake. 

 Solution:

Any object that is deleted in Active Directory will not instantly disappear, it will instead remain as a TombStoned object in the "Deleted Objects" container of Active Directory.  The object will remain there for the tombstone lifetime before it is permanently removed. The Active Directory tombstone lifetime is set to 180 days in EBS 2008 by default.

You can use this feature of Active Directory and first, re-animate the deleted computer object back. The re-animated object will preserve the same GUID, SID, class, SamAccountName, etc. and is effectively the same object, however, a few important attributes will be stripped from it after re-animation. The most important attribute we care about is the group membership attribute (memberOf). Since group membership is a property of the group the object belongs to and not the object itself, this attribute will be lost after reanimation. Hence, after re-animation we will apply the same group memberships to the computer object and proceed with replacement. Depending on the condition of your AD, EBS Messaging replacement may block after domain join, expecting an ntdsSettings server object in the configuration partition. If this happens, simply create a new ntdsSettings object as explained below.

More specifically, this computer object recovery is a three-step process.

1.       Re-animate the old deleted computer object:

There are many references online that explain how to recover an Active Directory object that has been tombstoned. I will not attempt to repeat the process here and instead will just include those references. The thing that you must remember is to restore the computer object back to the location it was at the end of EBS setup, which is in the Domain Controllers OU under the default domain partition.

Use the references below to manually restore the deleted EBS Messaging server.

-          Manually Undeleting Objects in Active Directory

https://www.petri.co.il/manually-undeleting-objects-windows-active-directory-ad.htm

-          Reanimating Active Directory Tombstone Objects

https://technet.microsoft.com/en-us/magazine/2007.09.tombstones.aspx?pr=blog

2.       Reapply the group memberships to the restored computer object:

As mentioned above, group memberships are stripped from the object after reanimations because group membership is an attribute on the group the object belongs to, not on the object. Hence, after the computer object is recovered, it must be decorated with at least the group memberships it had by the end of setup. Applying these group memberships will enable replacement to succeed. These groups are:          

Domain Controllers

Exchange Install Domain Servers

Exchange Servers

SCE Managed Computers (<ServerName>_MG)

Windows Essential Business Servers 

3.       If replacement blocks expecting an ntdsSettings object under configuration partition, create it. You do not need to restore the original object from tombstone, and can simply create a new one. To do this, use adsiedit and create a new AD object of class nTDSDSA under the location CN=<EBSMessagingServerName>CN=Servers,CN=<SiteName>,CN=Sites,CN=Configuration,DC=<DomainName>,DC=COM.

You do not need to apply any properties or create any replication channels on this new object; DCPromo will take care of all these settings. Note that the above steps can also be used in restoring any non-EBS Exchange server after an accidental deletion of the server object. The only difference is that you will not need to apply the groups that are EBS-specific (step 2 above).

 After these steps you are ready to run EBS server replacement.

Good luck restoring …

Alireza Farhangi