Philip Van de Vyver on SCOM and Azure Log Analytics

SCOM - OMS Gateway Monitoring Addendum Management Pack

Philip Van de Vyver — Tue, 24 Apr 2018 19:19:13 GMT

Sometimes, it's not clear how many clients that are reporting and if there are any errors with the OMS gateway. If you use SCOM, you can use it to visualize performance, event and health state of the OMS gateway. The OMS documentation is clear on what kind of performance counters and event data that we can use: https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-oms-gateway I created an addon Management Pack called "Microsoft System Center Advisor Monitoring Addendum OMS Gateway". The Management Pack contains the following:

State view

The Management Pack discovers an OMS Gateway Server class with a bunch of useful properties. There is one Monitor active on the OMS Gateway Server class that monitors the Service of the OMS Gateway.

Performance View

The Management Pack collects 4 performance counters with the following performance collection rules:

Name	Description
OMS Gateway/Active Client Connection	Number of active client network (TCP) connections
OMS Gateway/Error Count	Number of errors
OMS Gateway/Connected Client	Number of connected clients
OMS Gateway/Rejection Count	Number of rejections due to any TLS validation error

Event view

The Management Pack collects 6 events with the following event collection rules (no alerting):

Name	Description
OMS Gateway - Event Collection Event 105	Destination server is not in allowed list or the destination port is not secure port (443). Ensure that the MMA agent on your Gateway server and the agents that are communicating with the Gateway are connected to the same Log Analytics workspace. or ERROR TcpConnection – Invalid Client certificate: CN=Gateway Ensure that: - You are using a Gateway with version number 1.0.395.0 or greater. - The MMA agent on your Gateway server and the agents that are communicating with the gateway are connected to the same Log Analytics workspace.
OMS Gateway - Event Collection Event 106	The OMS Gateway only supports TLS 1.0, TLS 1.1 and 1.2. It does not support SSL. For any unsupported TLS/SSL protocol version, OMS Gateway generates event ID 106.
OMS Gateway - Event Collection Event 400	Any application error that does not have a specific ID.
OMS Gateway - Event Collection Event 401	Wrong configuration. For example: listenPort = "text" instead of an integer.
OMS Gateway - Event Collection Event 402	Exception in parsing TLS handshake messages
OMS Gateway - Event Collection Event 403	Networking error. For example: cannot connect to target server.

You can download the addon Management Pack over here: https://gallery.technet.microsoft.com/SCOM-OMS-Gateway-Addendum-61c0dc18 I've included the VSAE project over here: VSAE Project

Disclaimer:

All information on this blog is provided on an as-is basis with no warranties and for informational purposes only. Use at your own risk. The opinions and views expressed in this blog are those of the author and do not necessarily state or reflect those of my employer.

SQL Server 2016 Management Pack - MSSQL 2016: Service Pack Compliance - SP1

Philip Van de Vyver — Fri, 05 Jan 2018 14:57:29 GMT

Just a quick update on the SQL Server 2016 Management Pack: You might have experienced that no matter what version of SQL Server Service Pack you install, you get this alert: The reason for this alert is actually not the Management Pack, but the SQL Server installation not updating the registry key. When looking at the registry key responsible for the SP version, you'll notice that this one isn't updated with the SP version number: A workaround can be to update the registry value to the according SP1 level "1". HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL13.[YourSQLInstanceName]\Setup\SP Or Override the monitor to either disable it or set 0 as the minimal required SP level. As stated by the SQL Release team, the issue will be fixed in the next SP2 release for SQL Server 2016: https://blogs.msdn.microsoft.com/sqlreleaseservices/released-system-center-management-pack-for-sql-server-and-dashboards-6-7-20-0/ Meanwhile, I created a SCOM PowerShell recovery task on the "Service Pack Compliance" Monitor that updates the registry key if necessary for the instance:

Download

"Microsoft SQL Server 2016 (Monitoring) - Addendum - SP1 Recovery" Management Pack: Microsoft.SQL_.Server.Monitoring.Addendum.SP_.Recovery Disclaimer: All information on this blog is provided on an as-is basis with no warranties and for informational purposes only. Use at your own risk. The opinions and views expressed in this blog are those of the author and do not necessarily state or reflect those of my employer.

PowerShell Alert Update Connector and Orchestrator

Philip Van de Vyver — Fri, 15 Dec 2017 19:00:56 GMT

Many of my customers have the need to "sort" SCOM alerts to different support teams. In the past, the Alert Update Connector was used: https://blogs.technet.microsoft.com/kevinholman/2012/09/29/opsmgr-public-release-of-the-alert-update-connector/ There still are some versions available: https://blogs.technet.microsoft.com/markmanty/2017/04/24/scom-alert-update-connector-example-for-scom-2016/ https://blogs.technet.microsoft.com/markmanty/2012/05/03/scom-alert-updater-service-connector-example-updating-scom-alerts/ The "Supported" way is to use System Center Orchestrator to process your alerts. For my customer I reused this example: sc.scomurr.com/scom-2012-flagging-alerts-for-processing-using-scorch-2012-2/ Here's an overview of what needs to be implemented:

Alert Enrichment Database and Table

In order to store the information of each alert. We need to create a database and table. (In my case a CE_Enrichment database and CE_Enrichment table) The .sql file can be downloaded on the bottom of this article. The following columns are added to the table:

Alert Enrichment Configuration Tool

Configuration

In order to get the tool working, we need to define a few settings in the "AE_ConfigurationTool.config" XML:

RootManagementServerEmulator
SQLServer
SQLDBName
SQLEnrichTable
SQLQuery
default_appid
default_owner
default_resstate
default_ticket

The Tool

I've extended Scott Murray's PowerShell script to do have a similar functionality as the Alert Update Connector. Multiple SCOM groups can be added to an alert, each group has it's own Custom Field definition. The advantage of the PowerShell tool is that you can customize the tool anyway you want. The code is open and there's no need to compile. The tool can be downloaded on the bottom of this article. Here's a screenshot of the tool :

System Center Operations Manager

In order to get the System Center Orchestrator working. We need to create some Resolution States. In the SCOM Console, goto Administration - Settings - Alerts - Alert Resolution States. Add the following Resolution States:

Alert Not SCORH Enriched
Alert SCORH Enriched
ProcessedBySCORCH

System Center Orchestrator

System Center Orchestrator will do the actual job:

Process Alerts with the Resolution State "New"
Set the Resolution State to "ProcessedBySCORCH"
Query the Database
Check if records for the Rule/Monitor exist in the database
- If exist:
  - Run PowerShell script and verify if the MonitoringObject is member of the groups defined in the table
  - Update with corresponding Custom Fields
  - Update Resolution State to "Alert SCORH Enriched"
- If not exist
  - Update Resolution State to "Alert Not SCORH Enriched"

Here's the screenshot of the Runbook that processes the alert: The tool can be downloaded on the bottom of this article. Because we need to connect to the management group in the PowerShell script we need to import the SCOM module. Orchestrator 2016 currently has an issue in loading 64bit modules. In order to let Orchestrator import 64bit modules change the following reg. key: HKLM\SOFTWARE\Wow6432Node\Microsoft\.NETFramework Reg_DWORD: OnlyUseLatestCLR Value: 1 Make sure that you know what you are doing before you break backward compatibility: https://social.technet.microsoft.com/Forums/lync/en-US/bda22a17-c274-4799-9624-97588fdf4357/orchestrator-2016-powershell-not-importing-modules?forum=scogeneral www.opsconfig.com/better-practices-orchestrator-automation-sometimes-that-blog-post-that-seems-too-good-to-be-true/

Download:

The Alert Enrichment Configuration tool: AE_ConfigurationTool_v2_0_8 The Orchestrator Runbook: 2.8-AUC SQL create Enrichment Table: CE_Enrichment Disclaimer: All information on this blog is provided on an as-is basis with no warranties and for informational purposes only. Use at your own risk. The opinions and views expressed in this blog are those of the author and do not necessarily state or reflect those of my employer.

Undiscovering / Stop Monitoring SQL 2008/2012/2014/2016 Express Editions

Philip Van de Vyver — Tue, 07 Nov 2017 11:38:59 GMT

This post is an update of https://blogs.technet.microsoft.com/emreguclu/2014/09/12/undiscovering-stop-monitoring-sql-200520082012-express-editions/ which adds SQL 2014 and SQL 2016 and removes SQL 2005 support to existing mp as per requests of customers. If you have the older version please just download and import this one. For all details about the configuration of the Management Pack, please visit Emre's Blog. MP Download: SQLExpress.RemoveMP.xml

Healthservice Service SID Type

Philip Van de Vyver — Mon, 29 May 2017 21:48:01 GMT

Hi, While visiting customers, I'm often asked how to configure RunAs accounts for the SQL Management Pack. I love the new way using the Healthservice Service SID as Kevin Holman documented in his blog post over here: https://blogs.technet.microsoft.com/kevinholman/2016/08/25/sql-mp-run-as-accounts-no-longer-required-2/ I don't want to go over the configuration of his Management Pack again. I just want to show on how you can query the Healthservice SID setting. Let's start: We're using the SC.EXE tool to set the SID type for the Healthservice. For more information on Windows Service Hardening, take a look over here: https://blogs.technet.microsoft.com/askperf/2008/02/03/ws2008-windows-service-hardening/ To set the SID type to unrestricted, we use the following command:

sc sidtype HealthService unrestricted

Showing the following output: That's it, you can use the Service SID of the Healthservice as a logon in SQL Server What the command does, is changing the value of the registry key: "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HealthService\ServiceSidType" to 1: You can use the following commands to get more information of the Healthservice SID type setting: sc qsidtype healthservice

An example of an unconfigured "NONE" SID type:

An example of a configured "UNRESTRICTED" SID type: sc showsid healthservice

An example of an unconfigured "Inactive" status: An example of a configured "Active" status: This wil hopefully point you in the right direction when verifying the Healthservice Service SID.

SCOM Environment Classification

Philip Van de Vyver — Thu, 27 Apr 2017 13:56:27 GMT

I often hear from customers that they are connecting all there Environments (Test, Acceptance, Production) to one SCOM environment. As expected, SCOM will generate alerts for all environments without making a difference between the environment. Customers are asking if we can split up the Environment monitoring in the SCOM console. A sollution could be making manual groups for each of the class types or Windows Computer objects. I try to avoid manual work and maintenance, therefore I developed the following Management Pack in order to classify:

Windows Computer Objects
Other Objects from other classes

Here's a high level overview of what the Management Pack does:

Create new Class "Windows Computer Environment Class" with "Windows Computer" as the base class and polulate Class with all "Windows Computer" objects and add a discoverd registry property "Environment" based on a Registry Key.
Create Groups for each environment (Test, Acceptance, Production) and populate them with "Windows Computer" Objects.
Create Groups for each environment (Test, Acceptance, Production) and Class and populate the with the instances of the class selected.

I will use Visual Studio 2015 with Authoring Extensions (VSAE) to develop the Management Pack. (available here: https://www.microsoft.com/en-us/download/details.aspx?id=30169) VSAE has the advantage to work with "Snippets". I will be using them in order to simplify the development and to reproduce XML data. Don't be afraid of using Visual Studio, it's a great tool to develop Management Packs.

Let's go :

Create new Class "Windows Computer Environment Class" with "Windows Computer" as the base class and polulate Class with all "Windows Computer" objects and add a discoverd registry property "Environment" based on a Registry Key.

Let's start by creating a new Visual Studio Project: Add a new reference (Microsoft.SystemCenter.InstanceGroup.library.mp) can be found on the SCOM install media "ManagementPacks" folder: Create a new Class Windows Computer Environment: Here's the XML of the Class, creating the Class "Environment.Windows_Computer_Environment" with "Microsoft.Windows.Computer" as a base class and with a property "Environment.Windows_Computer_EnvironmentProperty". Windows Computer Environment Class Environment Add new Discovery "Windows Computer Environment Discovery" : Change the Discovery with following Properties: Here's the XML from the "Configuration XML" Property. The discovery will look for a registry key "SYSTEM\Configuration\Environment" and will populate the value in the "Environment" Property of the "Windows Computer Environment" Class: $Target/Property[Type="Windows!Microsoft.Windows.Computer"]/NetworkName$ Environment.Windows_Computer_Environment.EnvironmentPropertyRule SYSTEM\Configuration\Environment 1 1 86000 $MPElement[Name="Environment.Windows_Computer_Environment"]$

        $MPElement[Name="Environment.Windows_Computer_Environment"]/Environment.Windows_Computer_Environment.EnvironmentProperty$

$Data/Values/Environment.Windows_Computer_Environment.EnvironmentPropertyRule$ $MPElement[Name="Windows!Microsoft.Windows.Computer"]/PrincipalName$ $Target/Property[Type="Windows!Microsoft.Windows.Computer"]/PrincipalName$ This is the will finilize of the configuration of the "Windows Computer Environment" Class with the Property "Environment". The overall configuration of you Visual Studio Solution should more or less looking as follow: After you import the Management Pack, you can see the property "Environment" when you use the "Discovered Inventory" view Targetted to "Windows Computer Environment"

Create Groups for each environment (Test, Acceptance, Production) and populate them with "Windows Computer" Objects.

For this part I will use Visual Studio Snippets to reproduce XML data for each environment. Create a new Snippet Template called "Environment Group.templatesnippet": Here's the XML from the Snippet Template The XML will:

Create a class for the Environment Group
Populate the class with members that have a specific Environment Property.

Note that I use the following variables:

#text('EnvironmentName')# (Used as variable for the environment)
#alias('managementpackname')# (Used to get the alias of the referenced Management Pack)

Discovery $MPElement$ $MPElement[Name="Environment.#text('EnvironmentName')#.Group"]$ $MPElement[Name="Environment.Windows_Computer_Environment"]$

              $MPElement[Name="#alias('Microsoft.SystemCenter.InstanceGroup.Library')#!Microsoft.SystemCenter.InstanceGroupContainsEntities"]$

                    $MPElement[Name="Environment.Windows_Computer_Environment"]/Environment.Windows_Computer_Environment.EnvironmentProperty$

Equal #text('EnvironmentName')# Environment - #text('EnvironmentName')#

Populate Environment - #text('EnvironmentName')# This discovery rule populates the group 'Environment - #text('EnvironmentName')# '

Next would be creating the Snippet Data where we reference the created Snippet Template ("Environment Group"): You now can define the variables used for the Snippet Template (in this case only EnvironmentName is used). Simply enter the Environment names for all environments. By doing this you will reproduce the XML for each defined Environment variable. This is showing how easy and fast you can develop parts of a Management Pack. The overall configuration of you Visual Studio Solution should more or less looking as follow: When you import the Management Pack you should see 3 groups created, one for each created environment (TST, ACC, PROD):

Environment - TST
Environment - ACC
Environment - PROD

Create Groups for each environment (Test, Acceptance, Production) and Class and populate the with the instances of the class selected.

Also for this part I will use Visual Studio Snippets to reproduce XML data for each environment. Create a new Snippet Template called "Environment Instance Group.templatesnippet" as shown earlier. Here's the XML of the Snippet Template. Note that I use the following variables:

#text('EnvironmentName')# (Used as variable for the environment)
#text('InstanceClass')# (Used as variable for the Instance Class we are selecting)
#text('InstanceClassMPAlias')# (Used as variable for the Alias of the Management Pack that holds Instance Class that we are selecting)
#alias('managementpackname')# (Used to get the alias of the referenced Management Pack)

Discovery $MPElement$ $MPElement[Name="Environment.#text('EnvironmentName')#.#text('InstanceClass')#.Group"]$ $MPElement[Name="#text('InstanceClassMPAlias')#!#text('InstanceClass')#"]$

 $MPElement[Name="#alias('Microsoft.SystemCenter.InstanceGroup.Library')#!Microsoft.SystemCenter.InstanceGroupContainsEntities"]$

$MPElement[Name="Environment.#text('EnvironmentName')#.Group"]$ Environment - #text('EnvironmentName')# - #text('InstanceClass')# Group Populator This discovery rule populates the group 'Environment - #text('EnvironmentName')# - #text('InstanceClass')#' Environment - #text('EnvironmentName')# - #text('InstanceClass')# Create new Snippet Data called "Environment Instance Groups.mpsd" and reference the Snippet Template "Environment Instance Group" created in previous step. For this example I'll use the "Microsoft.Windows.Server.10.0.OperatingSystem" Class from the "Microsoft.Windows.Server.2016.Discovery" Management Pack (Windows Server 2016 Operating System) and populate all instance from a particular Environment to a Group. Therefore, we need to create a new reference to the "Microsoft.Windows.Server.2016.Discovery" Management Pack: Note the Alias for the Management Pack in the Properties of the Management Pack: Next would be creating the Snippet Data where we reference the created Snippet Template ("Environment Instance Group") as shown earlier. You now can use the variables used for the Snippet Template (EnvironmentName, InstanceClass, InstanceClassMPAlias)

EnvironmentName (The Environment Variable we defined earlier (TST, ACC, PROD)
InstanceClass (The Class that we select to populate the group with, for this example we'll use "Microsoft.Windows.Server.10.0.OperatingSystem")
InstanceClassMPAlias (The Management Pack Alias for the Management Pack that holds the class, for this example we'll use "MWS2016")

By doing this you will reproduce the XML for each defined variable set. This will end the configuration of the Management Pack. When you import the Management Pack, you'll see the following Groups created: The "Environment - PROD - Microsoft Windows Server 10.0.OperatingSystem" will show all production Windows Server 2016 Operating System Objects: You can now use these groups to build dashboards or other purposes. Here's the Visual Studio Project: https://msdnshared.blob.core.windows.net/projects/Environment.zip Here's the Management Pack XML: https://msdnshared.blob.core.windows.net/projects/Environment.xml