Also, we cannot create a monitor for each service and target it across all computers as each service may be present on bunch of computers and not on others. Thus targeting unanimously will result in false alarms and again, we may need 30+ windows service monitors targeted to all windows computers which will create overhead on agents and thus on the computers running the agent.

So, What is the solution?

Optimal solution would be creating a single rule to monitor all automatic services in each computer and alert on those which are not running. This can be accomplished using Powershell script with property bag output.

The rule runs on each computer at specific time interval, creates property bags for each service which is set to automatic but not running and an alert is generated for each property bag.

A catch to note in this monitoring scenario is not to alert on services that are stopped only for a moment. To overcome the issue, we will use consolidator condition. So only if the service is failed for ‘n’ consecutive samples, we will alert.

This solution, though optimal pose another challenge – What if we do not want to monitor a service which is set to automatic in one or few of computers.

This can be handled using a centrally located file with details of service and the computers to be excluded from monitoring.

We will see how to construct the Management Pack XML to accomplish this. You can also create MP using Visual Studio, MP Studio or Authoring Console.

Step 1:

Add references to the Management pack.

 1 <ManagementPack ContentReadable="true" xmlns:xsd="https://www.w3.org/2001/XMLSchema" xmlns:xsl="https://www.w3.org/1999/XSL/Transform">
 2   <Manifest>
 3     <Identity>
 4       <ID>GKLab.Windows.Automatic.Service.Monitoring</ID>
 5       <Version>1.0.0.0</Version>
 6     </Identity>
 7     <Name>GKLab Windows Automatic Service Monitoring</Name>
 8     <References>
 9       <Reference Alias="SC">
10         <ID>Microsoft.SystemCenter.Library</ID>
11         <Version>6.1.7221.0</Version>
12         <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
13       </Reference>
14       <Reference Alias="Windows">
15         <ID>Microsoft.Windows.Library</ID>
16         <Version>6.1.7221.0</Version>
17         <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
18       </Reference>
19       <Reference Alias="Health">
20         <ID>System.Health.Library</ID>
21         <Version>6.1.7221.0</Version>
22         <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
23       </Reference>
24       <Reference Alias="System">
25         <ID>System.Library</ID>
26         <Version>6.1.7221.0</Version>
27         <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
28       </Reference>
29       <Reference Alias="Performance">
30         <ID>System.Performance.Library</ID>
31         <Version>6.1.7221.0</Version>
32         <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
33       </Reference>
34     </References>
35   </Manifest>

Step 2:

Now create a Powershell property bag probe script. The Powershell script fetches list for all services that are set to start automatic and checks for the current status. For each service that are set to Automatic but not running, a property bag is created.

To exclude some services from being monitored, a centrally located CSV file is used and the path of file is passed as parameter to the script. The script reads list of services to be excluded from monitoring from CSV file and compares it with the list of services in the target computer. The property bag for excludes services are not created.

 1 param (
 2     [string] $excludeservicelist
 3 )
 4 if (test-path $excludeservicelist) {
 5 write-eventlog -logname "Operations Manager" -Source "Health Service Script" -EventID 776 -Message "WindowsAutomaticServiceMonitoring.ps1 - Accessing Exclusion List CSV" -EntryType Information
 6     $contents = Import-Csv $excludeservicelist
 7 }
 8 $TargetComputer = hostname
 9 $api = New-Object -comObject 'MOM.ScriptAPI'
10 $auto_services = Get-WmiObject -Class Win32_Service -Filter "StartMode='Auto'"
11 foreach ($service in $auto_services)
12 {
13     $isExcluded = 0
14     $state = $service.state
15     $name = $service.DisplayName
16     If ($Contents){
17         $contents | ForEach-Object{
18             $ExcludeServiceDisplayName = $_.ServiceToExclude
19             $ExcludeComputerName = $_.ComputersToExclude
20             if (($name -match $ExcludeServiceDisplayName) -and (($TargetComputer -match $ExcludeComputerName) -or ($ExcludeComputerName -match "ALL_COMPUTERS"))){
21                 $isExcluded = 1
22                 #write-eventlog -logname "Operations Manager" -Source "Health Service Script" -EventID 777 -Message "WindowsAutomaticServiceMonitoring.ps1 - Excluded Service Name - $ExcludeServiceDisplayName, Excluded Computer Name - $ExcludeComputerName" -EntryType Information
23             }
24         }
25     }
26     if (($isExcluded -eq 0) -and ($state -eq "Stopped")){
27         #write-eventlog -logname "Operations Manager" -Source "Health Service Script" -EventID 778 -Message "WindowsAutomaticServiceMonitoring.ps1 - Windows Service set to Automatic but Not Running - $name" -EntryType Information
28         $bag = $api.CreatePropertyBag()    
29         $bag.AddValue("ServiceName", $name)
30         $bag.AddValue("Status", $state)
31         $bag
32     }
33 }

Create a data source module incorporating the above written Powershell script. We will use consolidator condition as discussed in solution part to alert only on valid service failures.

 1   <TypeDefinitions>
 2     <ModuleTypes>
 3       <DataSourceModuleType ID="GKLab.Windows.Auto.Service.Monitoring.DataSource" Accessibility="Internal" Batching="false">
 4         <Configuration>
 5           <xsd:element minOccurs="1" name="ExcludeServiceList" type="xsd:string" />
 6           <xsd:element minOccurs="1" name="IntervalSeconds" type="xsd:integer" />
 7           <xsd:element minOccurs="1" name="ConsolidationInterval" type="xsd:integer" />
 8           <xsd:element minOccurs="1" name="Count" type="xsd:integer" />
 9         </Configuration>
10         <OverrideableParameters>
11           <OverrideableParameter ID="IntervalSeconds" Selector="$Config/IntervalSeconds$" ParameterType="int" />
12           <OverrideableParameter ID="Count" Selector="$Config/Count$" ParameterType="int" />
13           <OverrideableParameter ID="ConsolidationInterval" Selector="$Config/ConsolidationInterval$" ParameterType="int" />
14         </OverrideableParameters>
15         <ModuleImplementation Isolation="Any">
16           <Composite>
17             <MemberModules>
18               <DataSource ID="Trigger" TypeID="System!System.SimpleScheduler">
19                 <IntervalSeconds>$Config/IntervalSeconds$</IntervalSeconds>
20                 <SyncTime>00:00</SyncTime>
21               </DataSource>
22               <ProbeAction ID="Probe" TypeID="Windows!Microsoft.Windows.PowerShellPropertyBagProbe">
23                 <ScriptName>WindowsAutomaticServicesMonitoring.ps1</ScriptName>
24                 <ScriptBody><![CDATA[
25 param (
26     [string] $excludeservicelist
27 )
28 if (test-path $excludeservicelist) {
29 write-eventlog -logname "Operations Manager" -Source "Health Service Script" -EventID 776 -Message "WindowsAutomaticServiceMonitoring.ps1 - Accessing Exclusion List CSV" -EntryType Information
30     $contents = Import-Csv $excludeservicelist
31 }
32 $TargetComputer = hostname
33 $api = New-Object -comObject 'MOM.ScriptAPI'
34 $auto_services = Get-WmiObject -Class Win32_Service -Filter "StartMode='Auto'"
35 foreach ($service in $auto_services)
36 {
37     $isExcluded = 0
38     $state = $service.state
39     $name = $service.DisplayName
40     If ($Contents){
41         $contents | ForEach-Object{
42             $ExcludeServiceDisplayName = $_.ServiceToExclude
43             $ExcludeComputerName = $_.ComputersToExclude
44             if (($name -match $ExcludeServiceDisplayName) -and (($TargetComputer -match $ExcludeComputerName) -or ($ExcludeComputerName -match "ALL_COMPUTERS"))){
45                 $isExcluded = 1
46                 write-eventlog -logname "Operations Manager" -Source "Health Service Script" -EventID 777 -Message "WindowsAutomaticServiceMonitoring.ps1 - Excluded Service Name - $ExcludeServiceDisplayName, Excluded Computer Name - $ExcludeComputerName" -EntryType Information
47             }
48         }
49     }
50     if (($isExcluded -eq 0) -and ($state -eq "Stopped")){
51         write-eventlog -logname "Operations Manager" -Source "Health Service Script" -EventID 778 -Message "WindowsAutomaticServiceMonitoring.ps1 - Windows Service set to Automatic but Not Running - $name" -EntryType Information
52         $bag = $api.CreatePropertyBag()    
53         $bag.AddValue("ServiceName", $name)
54         $bag.AddValue("Status", $state)
55         $bag
56     }
57 }
58              ]]></ScriptBody>
59                 <Parameters>
60                   <Parameter>
61                     <Name>ExcludeServiceList</Name>
62                     <Value>$Config/ExcludeServiceList$</Value>
63                   </Parameter>
64                 </Parameters>
65                 <TimeoutSeconds>300</TimeoutSeconds>
66               </ProbeAction>
67               <ConditionDetection ID="Consolidator" TypeID="System!System.ConsolidatorCondition">
68                 <Consolidator>
69                   <ConsolidationProperties>
70                     <PropertyXPathQuery>$Target/Property[Type="Windows!Microsoft.Windows.Computer"]/PrincipalName$</PropertyXPathQuery>
71                     <PropertyXPathQuery>Property[@Name='ServiceName']</PropertyXPathQuery>
72                   </ConsolidationProperties>
73                   <TimeControl>
74                     <WithinTimeSchedule>
75                       <Interval>$Config/ConsolidationInterval$</Interval>
76                     </WithinTimeSchedule>
77                   </TimeControl>
78                   <CountingCondition>
79                     <Count>$Config/Count$</Count>
80                     <CountMode>OnNewItemTestOutputRestart_OnTimerSlideByOne</CountMode>
81                   </CountingCondition>
82                 </Consolidator>
83               </ConditionDetection>
84             </MemberModules>
85             <Composition>
86               <Node ID="Consolidator">
87                 <Node ID="Probe">
88                   <Node ID="Trigger" />
89                 </Node>
90               </Node>
91             </Composition>
92           </Composite>
93         </ModuleImplementation>
94         <OutputType>System!System.ConsolidatorData</OutputType>
95       </DataSourceModuleType>
96     </ModuleTypes>
97   </TypeDefinitions>

Step 4:

Next we will create a rule using the data source. Below configuration needs to be customized according to the need.

ExcludeServiceList – the UNC path for the excluded services list file (in CSV format). Sample CSV provided below.

CSV has two headers- “ServiceToExclude” which is display name of service.

ComputersToExclude – NetBIOS Name of computer. If two or more computers, it can be specified as individual entry or using regular expression syntax. If need to exclude in all computers, the value should be “ALL_Computers”

1 ServiceToExclude,ComputersToExclude
2 Distributed Transaction Coordinator,SCOM2012R2
3 Windows Audio,Win2k12-DC
4 Remote Registry,ALL_Computers
5 Software Protection,SCOM2012R2|Win2k12-DC

IntervalSeconds – Polling Interval in Seconds

Count – Number of polls, the service should fail to alert. (Minimum 2)

ConsolidationInterval – The interval time within which the service status fails ‘n’ number of times to generate alert.  (Minimum value = (n-1) * IntervalSeconds where n = count)

 1   <Monitoring>
 2     <Rules>
 3       <Rule ID="GKLab.Windows.AutomaticService.Monitoring.Rule" Enabled="true" Target="Windows!Microsoft.Windows.Computer" ConfirmDelivery="true" Remotable="true" Priority="Normal" DiscardLevel="100">
 4         <Category>Alert</Category>
 5         <DataSources>
 6           <DataSource ID="DS" TypeID="GKLab.Windows.Auto.Service.Monitoring.DataSource">
 7             <ExcludeServiceList>\\SCOM2012R2\Configs\WindowsAutomaticServiceMonitoringExclusionList.csv</ExcludeServiceList>
 8             <IntervalSeconds>300</IntervalSeconds>
 9             <ConsolidationInterval>600</ConsolidationInterval>
10             <Count>2</Count>
11           </DataSource>
12         </DataSources>
13         <WriteActions>
14           <WriteAction ID="Alert" TypeID="Health!System.Health.GenerateAlert">
15             <Priority>1</Priority>
16             <Severity>2</Severity>
17             <AlertMessageId>$MPElement[Name="GKLab.Windows.AutomaticService.Monitoring.Rule.AlertMessage"]$</AlertMessageId>
18             <AlertParameters>
19               <AlertParameter1>$Data/Context/DataItem/Property[@Name='ServiceName']$</AlertParameter1>
20             </AlertParameters>
21             <Suppression>
22               <SuppressionValue>$Target/Property[Type="Windows!Microsoft.Windows.Computer"]/PrincipalName$</SuppressionValue>
23               <SuppressionValue>$Data/Context/DataItem/Property[@Name='ServiceName']$</SuppressionValue>
24             </Suppression>
25           </WriteAction>
26         </WriteActions>
27       </Rule>
28     </Rules>
29   </Monitoring>

Step 5:

Final step is to construct XML for presentation and language packs. Ensure the close the <ManagementPack> tag.

 1   <Presentation>
 2     <StringResources>
 3       <StringResource ID="GKLab.Windows.AutomaticService.Monitoring.Rule.AlertMessage" />
 4     </StringResources>
 5   </Presentation>
 6   <LanguagePacks>
 7     <LanguagePack ID="ENU" IsDefault="true">
 8       <DisplayStrings>
 9     <DisplayString ElementID="GKLab.Windows.Automatic.Service.Monitoring">
10           <Name>GKLab Windows Automatic Service Monitoring</Name>
11           <Description>GKLab Windows Automatic Service Monitoring Management Pack</Description>
12         </DisplayString>
13         <DisplayString ElementID="GKLab.Windows.Auto.Service.Monitoring.DataSource">
14           <Name>GKLab Windows Automatic Service Monitoring Data Source</Name>
15           <Description>GKLab Windows Automatic Service Monitoring Data Source</Description>
16         </DisplayString>
17         <DisplayString ElementID="GKLab.Windows.AutomaticService.Monitoring.Rule">
18           <Name>Windows Automatic Services Monitoring Rule</Name>
19           <Description>Windows Automatic Services Monitoring Rule</Description>
20         </DisplayString>
21         <DisplayString ElementID="GKLab.Windows.AutomaticService.Monitoring.Rule" SubElementID="Alert">
22           <Name>Alert</Name>
23         </DisplayString>
24         <DisplayString ElementID="GKLab.Windows.AutomaticService.Monitoring.Rule" SubElementID="DS">
25           <Name>GKLab Windows Automatic Service Monitoring Data Source</Name>
26         </DisplayString>
27         <DisplayString ElementID="GKLab.Windows.AutomaticService.Monitoring.Rule.AlertMessage">
28           <Name>Windows Automatic Services Monitoring Alert</Name>
29           <Description>Windows Service {0} is set to auto-start but is currently not running.</Description>
30         </DisplayString>
31       </DisplayStrings>
32     </LanguagePack>
33   </LanguagePacks>
34 </ManagementPack>

Step 7:

Deploy the MP in lab and check for alerts.

I have attached copy of XML which you can import in to any authoring tool. Customize as per your needs and have fun.

Happy SCOMing…

In this post we will discuss about Powershell Discovery from a centrally located configuration file (CSV format) with an example scenario involving TCP Port monitoring. We will also discuss the impact of this method on end users, IT/SCOM Administrators.

Basics - Classes, Objects, Targets and Discoveries:

An object is the basic unit of management in Operations Manager. An object typically represents something in your computing environment, such as a computer, a logical disk, or a database. A class represents a kind of object, and every object in Operations Manager is considered an instance of a particular class. A target in the Operations console represents all instances of a particular class. A discovery is a special kind of rule to populate the class with instances.

Scenario:

Consider a scenario where you have a datacenter with 1000+ Windows and Unix Servers. We as SCOM Administrators are requested to configure monitoring for various TCP ports from different watcher nodes across various servers in datacenter.

This can be accomplished using TCP Port Template in Authoring Pane of console. But the drawbacks of using this template are:

Each port for each server needs to be configured manually.

Each entry creates bunch of classes, groups, overrides and number of workflows increase which will impact SCOM performance.

There is no central configuration/information on what is being monitored and the monitoring criteria.

Future changes needs to be manually configured in console.

If a watcher node is decommissioned, each port monitored by the watcher node need to be moved to other watcher node manually.

Each time application team has a new request to add/delete or modify, SCOM administrator need to make changes. In real environment, this includes Change requests, approvals etc which can consume considerable time.

Effective Solution:

To overcome the issues, it would be better to have a configuration in a central location and pull the information to SCOM at regular intervals. This way, once the initial configuration is setup,

The application team can maintain the list and can follow their own approval process to add/modify/delete.

The list can be mass updated.

The addition/modification/deletion is automatically sync’ed with SCOM at regular intervals.

No new workflows are added to SCOM for every addition and hence the impact on SCOM performance is minimal. Thus with handful of monitors and rules 1000s of objects can be monitored.

The information is available centrally.

The monitoring solution is self supported and cost effective in terms of support hours.

Below is step by step process with xml fragments included. The entire MP XML file is attached to the blog which you can download and test it in your lab.

Step 1: Create a New Management Pack “GKLab.TCP.Port.Monitoring”

Step 2: Add “Microsoft.SystemCenter.SyntheticTransactions.Library.mp” as reference.

Here is XML fragment for Step 1 and Step 2

 1 <?xml version="1.0" encoding="utf-8"?><ManagementPack ContentReadable="true" SchemaVersion="2.0" OriginalSchemaVersion="1.0" xmlns:xsd="https://www.w3.org/2001/XMLSchema" xmlns:xsl="https://www.w3.org/1999/XSL/Transform">
 2   <Manifest>
 3     <Identity>
 4       <ID>GKLab.TCP.Port.Monitoring</ID>
 5       <Version>1.0.0.0</Version>
 6     </Identity>
 7     <Name>GKLab.TCP.Port.Monitoring</Name>
 8     <References>
 9       <Reference Alias="SystemCenter">
10         <ID>Microsoft.SystemCenter.DataWarehouse.Library</ID>
11         <Version>7.1.10226.0</Version>
12         <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
13       </Reference>
14       <Reference Alias="Windows">
15         <ID>Microsoft.Windows.Library</ID>
16         <Version>7.5.8501.0</Version>
17         <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
18       </Reference>
19       <Reference Alias="MicrosoftSystemCenterSyntheticTransactionsLibrary">
20         <ID>Microsoft.SystemCenter.SyntheticTransactions.Library</ID>
21         <Version>7.1.10226.1090</Version>
22         <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
23       </Reference>
24       <Reference Alias="Performance">
25         <ID>System.Performance.Library</ID>
26         <Version>7.0.8433.0</Version>
27         <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
28       </Reference>
29       <Reference Alias="System">
30         <ID>System.Library</ID>
31         <Version>7.5.8501.0</Version>
32         <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
33       </Reference>
34       <Reference Alias="SC">
35         <ID>Microsoft.SystemCenter.Library</ID>
36         <Version>7.0.8433.0</Version>
37         <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
38       </Reference>
39       <Reference Alias="Health">
40         <ID>System.Health.Library</ID>
41         <Version>7.0.8433.0</Version>
42         <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
43       </Reference>
44     </References>
45   </Manifest>

Step 3: Create a custom class “GKLab.TCP.Port.Monitoring.Class” to store TCP Port monitoring configuration. The base class is “Microsoft.SystemCenter.SyntheticTransactions.TCPPortCheckPerspective”

 1     <TypeDefinitions>
 2     <EntityTypes>
 3       <ClassTypes>
 4         <ClassType ID="GKLab.TCP.Port.Monitoring.Class" Accessibility="Internal" Abstract="false" Base="MicrosoftSystemCenterSyntheticTransactionsLibrary!Microsoft.SystemCenter.SyntheticTransactions.TCPPortCheckPerspective" Hosted="true" Singleton="false" Extension="false">
 5           <Property ID="ServerName" Type="string" AutoIncrement="false" Key="true" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
 6           <Property ID="Port" Type="int" AutoIncrement="false" Key="true" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
 7           <Property ID="NoOfRetries" Type="int" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
 8           <Property ID="TimeWindowInSeconds" Type="int" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" />
 9         </ClassType>
10       </ClassTypes>
11       </EntityTypes>

Step 4: Now we need to create discovery data source. Before that we will discuss the CSV file format we will be using to store the configuration data.

We will name it as “TCPPortMonitoringList.csv”. The CSV has ServerName, PortNumber, WatcherNode, IntervalSeconds, NoOfRetries and TimeWindowInSeconds as header.

ServerName – Monitored Server Name (NetBIOS or FQDN)

PortNumber – Port Number to be monitored in monitored server.

WatcherNode – Computer/SCOM Agent that needs to monitor the port in monitored server.

IntervalSeconds – Monitoring Interval in seconds.

NoOfRetries – Number of times the monitor should fail before the alert is generated. This will reduce the alerts generated due to network latency. (Minimum value – 2)

TimeWindowInSeconds – Total time interval within which the monitor has to fail to generate an alert. (Minimum value = IntervalSeconds)

Step 5: Since we will use Powershell Script discovery, create a custom data source with a System.SimpleScheduler module and a Microsoft.Windows.PowerShellDiscoveryProbe probe module.

Since we have a centralized configuration CSV file, we can run the discovery from any one management server and populate the objects. In SCOM 2012, we will target the discovery against All Management Server Resource Pool, so that anyone MS will pick up the workflow. The discovery is thus highly available. The CSV file path should be shared so that it can be accessed from any MS.

Below is XML fragment for Custom Discovery module with embedded Powershell script.

  1     <ModuleTypes>
  2       <DataSourceModuleType ID="GKLab.TCP.Port.Monitoring.Discovery.DataSource" Accessibility="Internal" Batching="false">
  3         <Configuration>
  4           <xsd:element minOccurs="1" name="IntervalSeconds" type="xsd:integer" xmlns:xsd="https://www.w3.org/2001/XMLSchema" />
  5           <xsd:element minOccurs="1" name="SyncTime" type="xsd:string" xmlns:xsd="https://www.w3.org/2001/XMLSchema" />
  6           <xsd:element minOccurs="1" name="filePath" type="xsd:string" xmlns:xsd="https://www.w3.org/2001/XMLSchema" />
  7         </Configuration>
  8         <OverrideableParameters>
  9           <OverrideableParameter ID="IntervalSeconds" Selector="$Config/IntervalSeconds$" ParameterType="int" />
 10           <OverrideableParameter ID="FilePath" Selector="$Config/filePath$" ParameterType="string" />
 11         </OverrideableParameters>
 12         <ModuleImplementation Isolation="Any">
 13           <Composite>
 14             <MemberModules>
 15               <DataSource ID="DS" TypeID="System!System.SimpleScheduler">
 16                 <IntervalSeconds>$Config/IntervalSeconds$</IntervalSeconds>
 17                 <SyncTime>$Config/SyncTime$</SyncTime>
 18               </DataSource>
 19               <ProbeAction ID="Probe" TypeID="Windows!Microsoft.Windows.PowerShellDiscoveryProbe">
 20                 <ScriptName>TCPPortMonitoringConfigDiscovery.ps1</ScriptName>
 21                 <ScriptBody>
 22 param(
 23 [string] $sourceId,
 24 [string] $managedEntityId,
 25 [string] $filePath  )
 26 
 27 #Initialize SCOM API 
 28 
 29 $api = new-object -comObject 'MOM.ScriptAPI'
 30 $discoveryData = $api.CreateDiscoveryData(0, $SourceId, $ManagedEntityId) 
 31 write-eventlog -logname "Operations Manager" -Source "Health Service Script" -EventID 999 -Message "TCP Port Monitoring: looking for CSV file" -EntryType Information
 32 # $filePath variable contains UNC path of CSV Config file
 33 if (test-path $filePath) {
 34     write-eventlog -logname "Operations Manager" -Source "Health Service Script" -EventID 999 -Message "TCP Port Monitoring: Accessing CSV file" -EntryType Information
 35        $contents = Import-Csv $filePath
 36        try{
 37        $Path = (Get-ItemProperty "HKLM:SOFTWARE\Microsoft\System Center Operations Manager\12\Setup\Powershell\V2").InstallDirectory
 38     $Path1 = $Path + "OperationsManager\OperationsManager.psm1"
 39     if (Test-Path $Path1)
 40     {
 41         Import-Module $Path1
 42     }
 43     else
 44     {
 45         Import-Module OperationsManager
 46     }
 47            New-SCOMManagementGroupConnection
 48         #Retrieve all windows computers which can be used as watcher nodes
 49            $allServers = Get-SCClass | where { $_.Name -eq ("Microsoft.Windows.Computer")} | get-scommonitoringobject
 50     }
 51     catch{
 52     write-eventlog -logname "Operations Manager" -Source "Health Service Script" -EventID 999 -Message "TCP Port Monitoring: $_" -EntryType Information
 53     }
 54     #Read line by line from configuration file and create instance of TCP Port Monitoring Class
 55     $contents | ForEach-Object{
 56         $ServerName = $_.ServerName
 57         $PortNumber = $_.PortNumber
 58         $WatcherNode = $_.WatcherNode
 59         $NoOfRetries = $_.NoOfRetries
 60         $TimeWindowInSeconds = $_.TimeWindowInSeconds
 61         $Config = "$ServerName"+":"+"$PortNumber" # Will be used as display name
 62         write-eventlog -logname "Operations Manager" -Source "Health Service Script" -EventID 555 -Message "Checking servers" -EntryType Information
 63         $allServers | ForEach-Object{
 64             #Create instance only if the watcher node is managed by SCOM as the instance will hosted by the watcher node.
 65             #The hosting object is windows computer whose display name is equal to watcher node value from CSV
 66             #If there is no matching windows computer managed by SCOM, then the instance cannot be hosted. Hence the instance is not discovered.
 67             if((($_.DisplayName).toLower()).contains($WatcherNode.toLower())){
 68                 write-eventlog -logname "Operations Manager" -Source "Health Service Script" -EventID 555 -Message "Creating Instance for $Config" -EntryType Information
 69                 $instance = $discoveryData.CreateClassInstance("$MPElement[Name='GKLab.TCP.Port.Monitoring.Class']$")
 70                 $instance.AddProperty("$MPElement[Name='GKLab.TCP.Port.Monitoring.Class']/ServerName$", $ServerName)
 71                 $instance.AddProperty("$MPElement[Name='GKLab.TCP.Port.Monitoring.Class']/Port$", $PortNumber)
 72                 $instance.AddProperty("$MPElement[Name='GKLab.TCP.Port.Monitoring.Class']/NoOfRetries$", $NoOfRetries)
 73                 $instance.AddProperty("$MPElement[Name='GKLab.TCP.Port.Monitoring.Class']/TimeWindowInSeconds$", $TimeWindowInSeconds)
 74                 #The hosting object is windows computer whose display name is equal to watcher node value from CSV
 75                 $instance.AddProperty("$MPElement[Name='Windows!Microsoft.Windows.Computer']/PrincipalName$", $_.DisplayName)
 76                 $instance.AddProperty("$MPElement[Name='System!System.Entity']/DisplayName$", $Config)
 77                 $discoveryData.AddInstance($instance)
 78                 return
 79             }
 80         }
 81     }
 82 }
 83 $discoveryData
 84 Remove-variable api
 85 Remove-variable discoveryData
 86 </ScriptBody>
 87                 <Parameters>
 88                   <Parameter>
 89                     <Name>sourceId</Name>
 90                     <Value>$MPElement$</Value>
 91                   </Parameter>
 92                   <Parameter>
 93                     <Name>managedEntityId</Name>
 94                     <Value>$Target/Id$</Value>
 95                   </Parameter>
 96                   <Parameter>
 97                     <Name>filePath</Name>
 98                     <Value>$Config/filePath$</Value>
 99                   </Parameter>
100                 </Parameters>
101                 <TimeoutSeconds>300</TimeoutSeconds>
102               </ProbeAction>
103             </MemberModules>
104             <Composition>
105               <Node ID="Probe">
106                 <Node ID="DS" />
107               </Node>
108             </Composition>
109           </Composite>
110         </ModuleImplementation>
111         <OutputType>System!System.Discovery.Data</OutputType>
112       </DataSourceModuleType>
113     </ModuleTypes>
114     </TypeDefinitions>

Step 6: Now that we have created discovery data source, we will create a discovery GKLab.TCP.Port.Monitoring.Discovery.

Below is the discovery xml fragment. The UNC Path is mentioned in filePath.

 1   <Monitoring>
 2     <Discoveries>
 3       <Discovery ID="GKLab.TCP.Port.Monitoring.Discovery" Enabled="false" Target="SC!Microsoft.SystemCenter.AllManagementServersPool" ConfirmDelivery="true" Remotable="true" Priority="Normal">
 4         <Category>Discovery</Category>
 5         <DiscoveryTypes>
 6           <DiscoveryClass TypeID="GKLab.TCP.Port.Monitoring.Class" />
 7         </DiscoveryTypes>
 8         <DataSource ID="DS" TypeID="GKLab.TCP.Port.Monitoring.Discovery.DataSource">
 9           <IntervalSeconds>500</IntervalSeconds>
10           <SyncTime>00:00</SyncTime>
11           <filePath>\\SCOM2012R2\Configs\TCPMonitoringConfig.csv</filePath>
12         </DataSource>
13       </Discovery>
14     </Discoveries>
15   </Monitoring>

Step 8: Add Language Packs and close the ManagementPack tag.

 1   <LanguagePacks>
 2     <LanguagePack ID="ENU" IsDefault="true">
 3       <DisplayStrings>
 4         <DisplayString ElementID="GKLab.TCP.Port.Monitoring">
 5           <Name>GKLab TCP Port Monitoring</Name>
 6           <Description>This Management pack monitors the list of ports discovered from config file.</Description>
 7         </DisplayString>
 8         <DisplayString ElementID="GKLab.TCP.Port.Monitoring.Class">
 9           <Name>GKLab TCP Port Monitoring Class</Name>
10           <Description>Class Contains Instances of TCP Ports that needs to be monitored from specific watcher nodes</Description>
11         </DisplayString>
12         <DisplayString ElementID="GKLab.TCP.Port.Monitoring.Class" SubElementID="NoOfRetries">
13           <Name>No Of Retries</Name>
14         </DisplayString>
15         <DisplayString ElementID="GKLab.TCP.Port.Monitoring.Class" SubElementID="Port">
16           <Name>Port</Name>
17         </DisplayString>
18         <DisplayString ElementID="GKLab.TCP.Port.Monitoring.Class" SubElementID="ServerName">
19           <Name>Server Name</Name>
20         </DisplayString>
21         <DisplayString ElementID="GKLab.TCP.Port.Monitoring.Class" SubElementID="TimeWindowInSeconds">
22           <Name>Time Window In Seconds</Name>
23         </DisplayString>
24         <DisplayString ElementID="GKLab.TCP.Port.Monitoring.Discovery">
25           <Name>GKLab TCP Port Monitoring Discovery</Name>
26           <Description>Discovers TCP Port Monitoring Configs from given CSV file.</Description>
27         </DisplayString>
28         <DisplayString ElementID="GKLab.TCP.Port.Monitoring.Discovery.DataSource">
29           <Name>GKLab TCP Port Monitoring Discovery Data Source</Name>
30           <Description>Data Source used by TCP Port Monitoring Discovery Rule</Description>
31         </DisplayString>
32       </DisplayStrings>
33     </LanguagePack>
34   </LanguagePacks>
35 </ManagementPack>

Step 9: Now import the management pack in SCOM and check if the configuration from CSV are discovered.

Go to Discovered Inventory in SCOM Console and change target to “GKLab TCP Port Monitoring Class” to view the discovered items.

Step 10: Now you can develop custom monitors and rules targeting this class.

Thus the entire configuration can be maintained in a CSV file located in a shared location. For any new or modification in requirement, the CSV file can be updated accordingly. There is no changes required in SCOM side unless any additional headers are added and need to be absorbed in SCOM.

I will post details about monitors and rules for TCP Port monitoring in future posts.

Happy SCOMing!!!

GKLab.TCP.Port.Monitoring.xml

There are multiple scenarios as a part of AD management where we need to retrieve Group Policy information for managed computers. There are generally two methods to get the information.

Method 1:

Most common method is to use gpresult.exe command which is detailed in this technet article. This works well only if that User executing the command has logged-in once at-least in the target computer. Else it throws below error.

The user does not have RSOP Data

Method 2:

Method 2 is to use Get-GPResultantSetOfPolicy PowerShell command-let which is detailed here. This command also works similar to Method 1 and requires User to login at-least once.

Using Method 1 and Method 2, even if we want the group policy information only for the computer irrespective of user, it is not possible without the user logged in at-least once as the command retrieves resulting set of policies that are enforced for specified user on the target computer.

Solution:

To overcome these issues, using Group Policy Management COM Object which is the base for gpresult.exe and Get-ResultantSetOfPolicy PS command-let serves better. We can use the COM object in VB or PS scripting. Here we will discuss about using it in PS Scripting.

#Initialize Variables

$OutputFile = “C:\Temp\GPOExport.html”

$ComputerName = “test.contoso.com”

$UserName = “john”

The first thing we do is create an instance of the GPMgmt.GPM object. We can use this object if the Group Policy Management Console is installed in the computer.

$gpm = New-Object -ComObject GPMgmt.GPM

Next step is to obtain all constants and save it in a variable.

$constants = $gpm.GetConstants()

Now create reference RSOP object using required constants.

$gpmRSOP = $GPM.GetRSOP($Constants.RSOPModeLogging,$null,0)

Next step is to specify Target Computer and User.

$gpmRSOP.LoggingComputer = $ComputerName

$gpmRSOP.LoggingUser = $UserName

Note: If we need the RSOP data for only Computer without considering User imposed Group Policy data, we need to use “RsopLoggingNoUser” constant value instead of $gpmRSOP.LoggingUser.

$gpmRSOP.LoggingFlags = $Constants.RsopLoggingNoUser

Next step is to query the target computer for RSOP GPO data.

$gpmRSOP.CreateQueryResults()

To export data to a output file below command is used.

HTML:

$gpmRSOP.GenerateReportToFile($constants.ReportHTML,$outputfile)

XML:

$gpmRSOP.GenerateReportToFile($constants.ReportXML,$outputfile)

Thus using GPMgmt.GPM COM object, we can obtain Resulting Set of Group Policies for Target Computer with or without considering the User and also without requirement of user logging at-least once.

Happy Scripting :-)

THE PROBLEM:

But the uniqueness however  in this requirement is that the computer where the Shell Command needs to be executed and the Target Server are not the same.

As Illustrated above, The monitor is targeted to a "Class A" which has several instances including "Server A". The Shell Command however needs to be executed on "Server B" and the results needs to be manipulated and the state of "Monitor A" needs to be set.

THE SOLUTION:

The "Microsoft.Unix.WSMan.Invoke.ProbeAction" probe based on which the Unix/Linux monitoring data sources are built in SCOM , has a parameter called "TargetSystem".

In normal scenarios, the value would always be "$Target/Property[Type="MicrosoftUnixLibrary!Microsoft.Unix.Computer"]/NetworkName$". Thus the Shell Command would be executed in "Server A" and the "Monitor A" would have a state based on results.

But the value of "TargetSystem" certainly need not to be Target Server's name. This can be changed to any server which has a SCOM Unix Agent installed with valid certificate for authentication. Then the Shell Command will be executed in the "Server B" rather than in Target Server.

Additionally you can pass the Target Server's name along with Shell Command as parameter if you have used the Promote Option for Shell Command

Find the XML code in PDF file here -- [View:~/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-01-04-34/1033.Example.Unix.ShellCommand.Monitoring.pdf:550:0]

Have great SCOMing!!!

Happy Scripting..

It is quite difficult to work with the Offline Unix agents especially in a large monitoring environment. Though SCOM offers native heartbeat monitor, it is hard to quickly determine whether the computer is actually down or something wrong with the agent configuration.

An Unix agent may be down due to various reasons like issue with SCX process not running or a run as account password got changed or certificate got reset or the computer might be down. SCOM has “UNIX/Linux Heartbeat Monitor”, “WS-Management Run As Account Health” and “WS-Management Certificate Health” monitors to monitor each of above mentioned criteria and alert for offline agents. But it would be tedious job for support guy to handle multiple alerts for same issue and correlating them to fix the agent which may cost considerable time.

Will it not be easy to have only one alert in case of heartbeat failure with the status of all other monitors in the summary?

But wait, should we also track down the ping status in the alert summary so that the support guy knows what he should do first?

Yes, that’s what we are going to do now using PowerShell. The below script can be scheduled to run in any management server and it logs event in “Operations Manager” event log.

You can create a event collection rule targeting the management server to look for the events and create an alert. The alert will indicate the agent which is offline and details of other monitors related with the issue.

Happy SCOMing!!!