1. Import the Management Pack
2. Wait a little while for the Discovery to happen
3. Go to the Agent Management Groups\Agents view in the SCOM Console to see the list of Agents and the Management Groups they are configured to communicate with.
4. Use the Agent Management Group Tasks in the Tasks pane to Add or Remove Management Groups.

My customer has a lot of manually installed SCOM Agents that have not yet been approved in the console, and wanted a quick way to approve only the ones that are listed in a text file.

The following PowerShell command can take care of this pretty easily (modify the C:\Servers.txt path as needed). Also, be sure that the servers are in FQDN format in the text file (Server.Domain.com).

Foreach ($Computer in (Get-Content C:\Servers.txt)) {Get-SCOMPendingManagement | where {$_.AgentPendingActionType -eq "ManualApproval" -and $_.AgentName -eq $Computer}| Approve-SCOMPendingManagement}

I researched this a bit and found that unused space in the Health Service store is not cleaned up automatically. To clean it up and reduce the size of the file, you can do an offline defragmentation of the file by doing the following:

1. Stop the Microsoft Monitoring Agent service (HealthService)
2. Open an admin command prompt and run the following command (modify the file path as needed)

   esentutl /d "C:\Program Files\Microsoft Monitoring Agent\Agent\Health Service State\Health Service Store\HealthServiceStore.edb"
   
   

3. Restart the Microsoft Monitoring Agent service

Doing this in my test environment will generally reduce the file size from ~110mb to ~25mb

To help my customer detect and automate this, I’ve created the attached Management Pack to monitor, collect, and reduce the size of the Health Service store file.

 The Management Pack contains:

• A Monitor to check the size of the Health Service database file
  • Monitor name is “Health Service Store File Size Check”
  • Targeted at the “Agent” class
  • Disabled by default
  • Warning Alert/State Change is generated if the file size is greater than the defined threshold
  • File size threshold is 100mb by default and can be configured via override
• A Rule to collect the size of the Health Service database file
  • Rule name is “Collection: Health Service Database File Size”
  • Targeted at the “Agent” class
  • Collected performance counter is
    • Object: Health Service Database
    • Counter: File Size
    • Instance: <Path to health service store file>
• A Task to manually do an offline defrag of the Health Service database file
  • Task name is “Health Service Database Offline Defrag”
  • Targeted at the “Agent” class
• A Recovery to automatically do an offline defrag of the Health Service database file when the monitor detects the size is over the defined threshold
  • Disabled by default

Details of the workflows

• The Monitor runs a PowerShell script that gets the Health Service State directory from the registry, then checks the size of the HealthServiceStore.edb file and compares against the defined threshold. The script returns a Property Bag to SCOM with the file path, size, threshold, and status (above/below threshold).
• The Collection Rule uses the same script as the Monitor, and just uses the file size property and maps it to performance counter data.
• The offline defrag Task and Recovery use a VBScript based on Matt Taylor’s task to restart SCOM Health Service script…all I did is modify it to run the offline defrag before restarting the service.

Screen shots from the Management Pack

Monitor

Rule

Task

Health Explorer

Alert

Performance Collection

HealthServiceStore.Monitoring.zip

After verifying with my customer that the Exchange 2010 Monitors were generating state changes, but no alerts were coming in, we reviewed the event log for the Correlation Engine (it logs to the Application Event log on the server it is installed on) and saw event 717 being logged every 5 minutes (more details below).

After doing some research, I found the following information which resolved the problem (provided by Microsoft Escalation Engineer Manoj Parvathaneni).

Quick Summary:
We had to install the Correlation Engine on a server without any SCOM 2012 server components installed, and include some SCOM 2007 dlls in the Correlation Engine installation directory.

Symptoms

When the Exchange Correlation Engine is installed on a System Center 2012 Operations Manager Management Server, the Automatic Alert Resolution feature in the Exchange Correlation Engine ceases to function and the following event is logged in the Application event log on the Management Server:

Log Name: Application
Source: MSExchangeMonitoringCorrelation
Event ID: 717
Task Category: General
Level: Warning
Keywords: Classic
User: N/A
Description:
Connection with the Operations Manager Root Management Server failed.

Error: System.InvalidCastException: [A]System.Collections.Generic.List`1[Microsoft.EnterpriseManagement.Monitoring.MonitoringAlertUpdateStatusIndigo] cannot be cast to [B]System.Collections.Generic.List`1[Microsoft.EnterpriseManagement.Monitoring.MonitoringAlertUpdateStatusIndigo]. Type A originates from 'mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' in the context 'LoadNeither' at location 'C:\Windows\assembly\GAC_64\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll'. Type B originates from 'mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' in the context 'LoadNeither' at location 'C:\Windows\assembly\GAC_64\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll'.
at Microsoft.EnterpriseManagement.Utilities.GetDeserializeObject[T](Byte[] serializedObject)
at Microsoft.EnterpriseManagement.DataAbstractionLayer.InstanceSpaceOperations.UpdateAlerts(Byte[] alerts, String comments, Nullable`1 modifyingConnectorId)
at Microsoft.EnterpriseManagement.Monitoring.MonitoringAlert.UpdateAlertsInternal[T](IList`1 alerts, String comments, Nullable`1 modifyingConnectorId, ManagementGroup managementGroup)
at Microsoft.EnterpriseManagement.Monitoring.MonitoringAlert.UpdateInternal(String comments, Nullable`1 modifyingConnectorId)
at Microsoft.EnterpriseManagement.Monitoring.MonitoringAlert.Update(String comments)
at Microsoft.Exchange.Monitoring.CorrelationEngine.MomSdkProxy.ResolveEntityAlert(Entity entity, MonitoringAlert monAlert)
at Microsoft.Exchange.Monitoring.CorrelationEngine.MomSdkProxy.ResolveEntityAlerts(Entity entity, String alertCriteriaString)
at Microsoft.Exchange.Monitoring.CorrelationEngine.CorrelationEngine.CorrelateWithinEntity(Node entity)
at Microsoft.Exchange.Monitoring.CorrelationEngine.CorrelationEngine.CorrelateBatchTask(Object batchData)

Number of occurrence: 3

You'll also notice the following error in the Exchange Correlation Engine logs:

2013-11-18T08:16:15.673Z,4,Information,"CorrelateBatchTask: caught exception [System.InvalidCastException: [A]System.Collections.Generic.List`1[Microsoft.EnterpriseManagement.Monitoring.MonitoringAlertUpdateStatusIndigo] cannot be cast to [B]System.Collections.Generic.List`1[Microsoft.EnterpriseManagement.Monitoring.MonitoringAlertUpdateStatusIndigo]. Type A originates from 'mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' in the context 'LoadNeither' at location 'C:\Windows\assembly\GAC_64\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll'. Type B originates from 'mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' in the context 'LoadNeither' at location 'C:\Windows\assembly\GAC_64\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll'.    at Microsoft.EnterpriseManagement.Utilities.GetDeserializeObject[T](Byte[] serializedObject)    at Microsoft.EnterpriseManagement.DataAbstractionLayer.InstanceSpaceOperations.UpdateAlerts(Byte[] alerts, String comments, Nullable`1 modifyingConnectorId)    at Microsoft.EnterpriseManagement.Monitoring.MonitoringAlert.UpdateAlertsInternal[T](IList`1 alerts, String comments, Nullable`1 modifyingConnectorId, ManagementGroup managementGroup)    at Microsoft.EnterpriseManagement.Monitoring.MonitoringAlert.UpdateInternal(String comments, Nullable`1 modifyingConnectorId)    at Microsoft.EnterpriseManagement.Monitoring.MonitoringAlert.Update(String comments)    at Microsoft.Exchange.Monitoring.CorrelationEngine.MomSdkProxy.ResolveEntityAlert(Entity entity, MonitoringAlert monAlert)    at Microsoft.Exchange.Monitoring.CorrelationEngine.MomSdkProxy.ResolveEntityAlerts(Entity entity, String alertCriteriaString)    at Microsoft.Exchange.Monitoring.CorrelationEngine.CorrelationEngine.CorrelateWithinEntity(Node entity)    at Microsoft.Exchange.Monitoring.CorrelationEngine.CorrelationEngine.CorrelateBatchTask(Object batchData)].

In addition, the Exchange Correlation Engine will not report any new Alerts or report the alerts to SCOM 2012 inconsistently.

Resolution

The exception above can be resolved by using one of the following methods:

Method 1: Install Exchange Correlation Engine on a dedicated system without the SCOM 2012 Management Server component

1. Install Exchange Correlation engine on this machine.
2. Stop the Microsoft Exchange Monitoring Correlation service.
3. Copy SCOM 2007 R2 SDK binaries into the CE install directory (Program Files\Microsoft\Exchange Server\v14\Bin).
   • Microsoft.EnterpriseManagement.OperationsManager.Common.dll
   • Microsoft.EnterpriseManagement.OperationsManager.dll
   • MomBidLdr.dll

**UPDATE: Files attached to this blog

Where can you find the above binary files?

• The SCOM 2007 R2 SDK Binaries Microsoft.EnterpriseManagement.OperationsManager.Common.dll and Microsoft.EnterpriseManagement.OperationsManager.dll can be found on a system that has SCOM 2007 R2 Console in the following directory Program Files\System Center Operations Manager 2007\SDK Binaries.
• The MOmBidLdr.dll can be found can be found on a system that has SCOM 2007 R2 Console in the following directory Program Files\System Center Operations Manager 2007

Method 2: Disable the Automatic Alert Resolution Feature in the Correlation Engine

By following this approach the customer will lose the ability to resolve the alerts automatically via the Exchange Correlation Engine.

To disable Automatic Alert Resolution, perform the following steps:

1. Log on to the server that's hosting the Microsoft Exchange Monitoring Correlation Engine service.
2. Locate the Correlation Engine configuration file named Microsoft.Exchange.Monitoring.CorrelationEngine.exe.config. By default, the file is located in C:\Program Files\Microsoft\Exchange Server\V14\Bin\, where C:\ is the Exchange installation directory.
3. Open Microsoft.Exchange.Monitoring.CorrelationEngine.exe.config in a text editor such as Notepad
4. Locate the following line in the configuration file:
   <add key="AutoResolveAlerts" value="true" />
5. Change <add key="AutoResolveAlerts" value="true" /> to <add key="AutoResolveAlerts" value="false" />
6. Restart the Microsoft Exchange Monitoring Correlation Engine service.

More Information

Exchange Correlation Engine:
The Correlation Engine(CE) is a stand-alone Windows service that uses the Operations Manager SDK interface to first retrieve the health model (or instance space) and then process state change events. By maintaining the health model in memory, and processing state change events, the Correlation Engine is able to determine when to raise an alert based on the state of the system. The Correlation Engine solely relies on the Operations Manager SDK. It does not use the agent to perform any of the correlation and other aspects of its operations. For further details, you can also reference the Exchange 2010 Management Pack guide.

Automatic Alert Resolution feature in Exchange Correlation Engine:
The Automatic Alert Resolution feature automatically closes related alerts when the Exchange 2010 Management Pack determines that the underlying issue is no longer a problem. This feature is provided by the Correlation Engine, and is enabled by default. Using Automatic Alert Resolution can cause multiple alerts to be logged if the same alert is logged again for another instance of the problem before the associated ticket has been resolved by support teams.

You may also want to disable this feature under the following or other conditions:

• If you're using ticketing or another support system that wouldn't work correctly if alerts are automatically resolved.

If you're using a connector with Operations Manager 2007. A connector is a custom service or program that allows Operations Manager to communicate with external systems. For example, you may want to disable this feature if you're using a connector that allows an external application to track Exchange 2010 Management Pack alerts.

SCOM2007FilesForEXCHCorrelationEngine.zip

The report has the following fields

• Agent Name
• Agent Version
• Agent Version Number
• Patch List
• Management Group
• HS Availability
• HSOutage Last Modified
• Health Service Watcher Previous Day Healthy
• Health Service Watcher Previous Day Unhealthy
• Health Service Watcher Previous Day Maintenance
• Heartbeat Previous Day Healthy
• Heartbeat Previous Day Unhealthy
• Heartbeat Previous Day Maintenance
• OSVersion

The HS Availability field indicates whether the agent is currently connecting to SCOM. This is based this on data from the vHealthServiceOutage view in the data warehouse, which tracks when Health Service outages begin and end for an Agent. The “HSOutage Last Modified” field will tell you when the agent availability last changes.

The “Previous Day Healthy/Unhealthy/Maintenance” fields show the percentage of the previous day that the Health Service Watcher and Heartbeat monitor for the agent were in a Healthy/Unhealthy/Maintenance state. I do see occasional issues with these numbers being inaccurate, but these fields plus the HSOutage fields should give a pretty good indication of whether or not an agent is online.

The following parameters are available for filtering:

Agent Name

Include only agents that contain the text entered

Leave this blank to include all agents

Patch List

Include only Agents whose PatchList property contains the text entered

Leave this blank to include all agents

Health Service Availability

Select to include only Available agents, Unavailable agents, or both

Agent Version

Select to include/exclude SCOM 2007, SCOM 2012, or SCOM 2012 R2 Agents (all are selected by default)

Management Group

Select to include/exclude agents from specific Management Groups (all are included by default)

Agent Version and Health.rdl

When SCOM cannot resolve a performance counter, it will log one of two events:

• Event ID 10102 (could not resolve counter)
• Event ID 10103 (could not resolve counter instance)

I created a simple Management Pack with an Alert Rule to alert on these events. The attached MP contains one Rule with the following criteria:

Name: Alert on Missing Performance Counters
Target: Health Service
Criteria:
              Event log: Operations Manager
              Event Source: Health Service Modules
             Event ID:10102 / 10103

The Rule will generate an alert when event ID 10102 (could not resolve counter) or 10103 (could not resolve counter instance) are logged.  Alert suppression is enabled for the Object, Counter, and Instance parameters so it will generate a separate alert for each counter that cannot be resolved on a monitored computer.

The Rule is disabled by default…I would generally recommend only enabling it on computers that you are having problems with at first, then on others if needed.

Here is a screen shot of the Alert:

MissingPerfCounterAlerting.xml

Operations Console

Before installing the SCOM 2012 Operations Console, you will need to install SQL CLR Types and Report View 2012 Runtime:

· SQL CLR Types

X86 Package(SQLSysClrTypes.msi)
X64 Package (SQLSysClrTypes.msi)

· Report Viewer 2012 Runtime – Download here

Web Console

Before installing the SCOM 2012 Web Console role, you will need to install the IIS Role and a number of services.  Tim McFadden has documented the steps for doing this with PowerShell in Windows Server 2008 R2 and Windows Server 2012…links to Tim’s blog and overview of the steps below…

Windows Server 2008 R2

1. Run the following PowerShell commands to install the prerequisites:
   
   Import-Module ServerManager
   
   Add-WindowsFeature NET-Framework-Core,Web-Static-Content,Web-Default-Doc,Web-Dir-Browsing,Web-Http-Errors,Web-Http-Logging,Web-Request-Monitor,Web-Filtering,Web-Stat-Compression,AS-Web-Support,Web-Metabase,Web-Asp-Net,Web-Windows-Auth –restart
   
   
2. Run the following command from elevated command prompt to register the ASP .NET 4 with IIS.
   
   c:\windows\Microsoft.NET\Framework64\v4.0.30319\aspnet_regiis.exe –r
   
   
3. Run the following command to enable IIS to work with .net 4.0
   
   c:\windows\system32\inetsrv\appcmd set config /section:isapiCgiRestriction /[path=`'C:\Windows\Microsoft.NET\Framework64\v4.0.30319\aspnet_isapi.dll`'].allowed:True
   
   
4. Restart the server

Windows Server 2012

Run the following PowerShell commands to install the prerequisites:

Import-Module ServerManager

Add-WindowsFeature NET-Framework-Core,AS-HTTP-Activation,Web-Static-Content,Web-Default-Doc,Web-Dir-Browsing,Web-Http-Errors,Web-Http-Logging,Web-Request-Monitor,Web-Filtering,Web-Stat-Compression,AS-Web-Support,Web-Metabase,Web-Asp-Net,Web-Windows-Auth –restart

The Rules are targeted at DB Engine and alert on Event ID 1205 in the Application Event Log:

However, my customer generated a deadlock and no SCOM Alert was generated.  Looking in the Application Event Log on the SQL Server, we saw that the 1205 event was not logged.

After doing some digging, I found that that SQL Server does not log this event by default…which was confirmed by running Select * from sys.messages where message_id=1205 on the master database…the results showed is_event_logged=0:

To change this, we ran Exec sp_altermessage 1205, 'WITH_LOG', 'true' and verified the change (is_event_logged=1):

Now I generate a deadlock and get the 1205 event in the Application Event Log:

And I get an Alert from the SQL Server Management Pack:

To generate a deadlock, I used the steps documented here.

The MP that I wrote for this is attached

• The MP contains replicas of the “Available Megabytes of Memory” and “Total CPU Utilization Percentage” Monitors from the Windows Server MPs
• The only change is a modification to the VBScript to get the Top 5 processes and include them in the Property Bag and Alert.
• The memory monitor will show the top 5 instances of Process\Private Bytes
• The CPU monitor will show the top 5 instances of Process\% Processor Time
• The MP has separate Monitors for Server 2003, 2008, 2008 R2 and 2012

Screenshots of the alerts are below:

SAMPLE.Windows.Server.CPU_Memory.Monitoring.zip

Windows Server Operating System – Version 6.0.7230.0

• Bug fixed: Microsoft.Windows.Server.LogicalDiskDiscovery.Module.Type.vbs script does not discover logical disks with large disk size
• Update to support two configurable threshold values (waiters and timeouts) for triggering alert ‘MAX concurrent API Reached’

One change that may require some attention is the update to the “Max Concurrent API Monitor”.

In the previous version of the MP (6.0.7061.0), the Max Concurrent API monitor would run a script that looks at three Netlogon semaphore counters (Waiters, Holders, and Timeouts).  If any of them are greater than 0 and less than 4gb, then we generate an alert. Some customers were seeing false alerts from the Monitor due to some of the counters (especially Semaphore Holders) being greater than zero during non-problematic conditions.

In the new version of the Management Pack, the Max Concurrent API Monitor has the following changes:

• The Semaphore Holders counter has been removed from the criteria that generates the alert
• The Alert will only be generated if Semaphore Waiters or Timeouts exceed a defined threshold (instead of 0).  These thresholds can be configured as needed via Overrides.  The default thresholds are:

• Semaphore Waiters = 50
• Semaphore Timeouts = 2000
  

More information on the MaxConcurrentAPI setting can be found here.

Active Directory - Version 6.0.8293.0

• Issue fixed: AD-Trust Monitor does not come back to healthy state
• Issue fixed: AD_Database_and_Log.vbs does not support using ‘.’ as decimal sign for non-English account.

Cluster – Version 6.0.7230.0

• Issue fixed: Cluster 2008 MP does not collect certain performance metrics.

DNS – Version 7.1.10259.0

• Issue fixed: DNSMetrics2012R2Probe script can cause high CPU in MonitoringHost.exe

DHCP – Version 6.0.7230.0

• Various issues fixed.

MSMQ 5.0 (6.0.6709.88)
MSMQ 6.0  (7.0.8569.0)
MSMQ 6.3 (7.1.10109.0)

• Update to support monitoring workgroup machines