Importing the dashboard

• Enable Diagnostic Logging for your Application Gateways (below)
• Import the ARM template from here: https://github.com/iamrobdavies/MonitoringExamples/tree/master/ApplicationGateway/Dashboard

Enabling Diagnostic Logging

Dashboard Queries

Avg throughput per second (Mb)

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.NETWORK" and Category == "ApplicationGatewayPerformanceLog"
| summarize avg(throughput_d) by Resource, bin(TimeGenerated, 1m)
| extend ThroughputMb = (avg_throughput_d/1000)/1000
| project Resource, TimeGenerated, ThroughputMb
| render timechart

Unhealthy backend VM count

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.NETWORK" and Category == "ApplicationGatewayPerformanceLog"
| summarize max(unHealthyHostCount_d) by Resource, bin(TimeGenerated, 1m)
| render timechart

Healthy backend VM Count

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.NETWORK" and Category == "ApplicationGatewayPerformanceLog"
| summarize max(healthyHostCount_d) by Resource, bin(TimeGenerated, 1m)
| render timechart

Failed requests by API

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.NETWORK" and Category == "ApplicationGatewayAccessLog"
| where httpStatus_d >= 400
| summarize count() by requestUri_s, bin(TimeGenerated, 1m)
| render timechart

Avg Latency (ms) by AppGW

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.NETWORK" and Category == "ApplicationGatewayPerformanceLog"
| summarize avg(latency_d) by Resource, bin(TimeGenerated, 1m)
| render timechart

Requests per Min by API

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.NETWORK" and Category == "ApplicationGatewayAccessLog"
| summarize count() by requestUri_s, bin(TimeGenerated, 1m)
| render timechart

MyService API Requests per Min

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.NETWORK" and Category == "ApplicationGatewayAccessLog"
| where requestUri_s == "/"
| summarize count() by requestUri_s, bin(TimeGenerated, 1m)
| render timechart

Error count past hour by AppGW

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.NETWORK" and Category == "ApplicationGatewayAccessLog"
| where httpStatus_d >= 400
| summarize count() by httpStatus_d, Resource
| project httpStatus_d, Resource, count_

Avg Requests per min

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.NETWORK" and Category == "ApplicationGatewayPerformanceLog"
| summarize avg(requestCount_d) by Resource, bin(TimeGenerated, 1m)
| render timechart

Avg Failed Requests per min

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.NETWORK" and Category == "ApplicationGatewayPerformanceLog"
| summarize avg(failedRequestCount_d) by Resource, bin(TimeGenerated, 1m)
| render timechart

HTTP Error count per hour by API

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.NETWORK" and Category == "ApplicationGatewayAccessLog"
| where httpStatus_d >= 400
| summarize count(httpStatus_d) by httpStatus_d,requestUri_s, bin(TimeGenerated, 1h)
| order by count_httpStatus_d desc
| project httpStatus_d, requestUri_s, TimeGenerated, count_httpStatus_d

Successful request count

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.NETWORK" and Category == "ApplicationGatewayAccessLog"
| where httpStatus_d < 400
| summarize count() by httpStatus_d, Resource
| project httpStatus_d, Resource, count_

Failed requests by backend VM

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.NETWORK" and Category == "ApplicationGatewayAccessLog"
| where httpStatus_d >= 400
| parse requestQuery_s with * "SERVER-ROUTED=" serverRouted "&" *
| extend httpStatus = tostring(httpStatus_d)
| summarize count() by serverRouted, bin(TimeGenerated, 5m)
| render timechart

Successful requests by backend VM

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.NETWORK" and Category == "ApplicationGatewayAccessLog"
| where httpStatus_d < 400
| parse requestQuery_s with * "SERVER-ROUTED=" serverRouted "&" *
| extend httpStatus = tostring(httpStatus_d)
| summarize count() by serverRouted, bin(TimeGenerated, 5m)
| render timechart

HTTP 502 errors by backend VM

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.NETWORK" and Category == "ApplicationGatewayAccessLog"
| where httpStatus_d == 502
| parse requestQuery_s with * "SERVER-ROUTED=" serverRouted "&" *
| extend httpStatus = tostring(httpStatus_d)
| summarize count() by serverRouted, bin(TimeGenerated, 5m)
| render timechart

Monitored AppGW List

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.NETWORK" and Category == "ApplicationGatewayAccessLog"
| distinct Resource, ResourceGroup

•        New incidents are waiting in the Tier 1 Support Group
•        This script runs & looks up who is in an AD Group for the Tier 1 Support Group
•        We then look at how many incidents are assigned to each Tier 1 analyst, to see who has the least work
•        The incident is assigned to the analyst with the fewest assigned active incidents
•        If >1 Analyst have the fewest number of active incidents, it’s assigned to whoever has gone longest since last having an incident assigned
•        You can configure a maximum Incident assignment limit, e.g. 10 active incidents per analyst.  If all analysts are full, then incidents wait in the Tier 1 Support Group
•        Optionally, you can enable ‘Auto Escalation’. If all Tier 1 analysts are at capacity & there are more unassigned active Tier 1 Incidents, they can be automatically bumped to another support Group, e.g. Tier 2.

####################################################################################################################### # Assigns active incidents for a particular Support Group to the members of an AD Group # Author: Rob Davies (Microsoft) # Disclamer: This program source code is provided "AS IS" without warranty representation or condition of any kind # either express or implied, including but not limited to conditions or other terms of merchantability and/or # fitness for a particular purpose. The user assumes the entire risk as to the accuracy and the use of this # program code. # Tested with System Center 2012 R2 Service Manager with PowerShell v4 # Name: Assign-SCSMIncidents.ps1 # v1.0 - 7/3/2015 - Initial Release # v1.1 - 7/12/2016 - Updates for Azure Automation #######################################################################################################################

param ( [string]$SCSMManagementServer="localhost" )

if (!(Get-module System.Center.Service.Manager)) { $scsmRegKey = 'HKLM:\SOFTWARE\Microsoft\System Center\2010\Service Manager\Setup' Import-Module "$((Get-ItemProperty -Path $scsmRegKey -Name InstallDirectory).InstallDirectory)\PowerShell\System.Center.Service.Manager.psd1" -ErrorAction Stop } if (!(Get-module ActiveDirectory)) {import-module ActiveDirectory -ErrorAction Stop }

$smCreds = Get-AutomationPSCredential -Name 'contoso-rob'

#============================================================================ #TODO: Set supportGroup as the DISPLAY NAME of the Support Group to monitor $supportGroup = "Tier 1" #TODO: Set escalationGroup as the DISPLAY NAME of the support group to escalate to when the supportGroup is full $escalationGroup = "Tier 2" #TODO: Distinguished Name of AD Group and find members: $ADGroupName = "CN=Tier1,OU=Workshop,DC=contoso,DC=work" #TODO: Max number of active incidents assigned to operator: $maxAssignedIncidents = 7 #TODO: Auto-Escalate. If $True enables the ability to esclate to escalationGroup. Either: $True or $False $autoEscalate = $False #============================================================================

#Work Item Assigned Relationship Type $WIAssigned = Get-SCRelationship -Name "System.WorkItemAssignedToUser" -ComputerName $SCSMManagementServer -Credential $smCreds

function whoToAssignTo ($helpdeskUserList) { #Get the user with the fewest number of active incidents assigned $numberOfAnalystsWithFewestIncidents=0 $firstUser = $helpdeskUserList | Sort-Object AssignedIncidents | Select-Object -First 1

    #Find out if there are any other users with the same number of active assigned incidents foreach ($x in $helpdeskUserList) { if ($x.AssignedIncidents -eq $firstUser.AssignedIncidents) {$numberOfAnalystsWithFewestIncidents++} } $availableUser = $helpdeskUserList | Sort-Object AssignedIncidents | Select-Object -First $numberOfAnalystsWithFewestIncidents

    #If there is >1 user with fewest number of assigned active Incidents if ($availableUser.Count -gt 1){

        #Find the incidents which was last assigned for each user $recentIncidents = @() foreach ($x in $availableUser) { $mostRecentIR = $x.ListOfIncidents | Sort-Object LastAssigned -Descending | select -First 1 $recentIncidents = $recentIncidents + $mostRecentIR }

        #What if there are multiple users with 0 incidents if($recentIncidents.Count -gt 0) { #Find the incident which was assigned the longest time ago. Then find the user who is assigned to this $mostRecentIRFilter = "Id -eq " + ($recentIncidents | Sort-Object LastAssigned | select -First 1).id $mostRecentIR = Get-SCClassInstance -Class (Get-SCClass -Name System.WorkItem.Incident -ComputerName $SCSMManagementServer -Credential $smCreds) -Filter $mostRecentIRFilter -ComputerName $SCSMManagementServer -Credential $smCreds $userLastAssignedAnIncident = Get-SCClassInstance -Id (Get-SCRelationshipInstance -SourceInstance $mostRecentIR -ComputerName $SCSMManagementServer -Credential $smCreds |?{$_.RelationshipId -eq $WIAssigned.Id -and $_.IsDeleted -eq $false}).TargetObject.Id -ComputerName $SCSMManagementServer -Credential $smCreds

            #return the user to assign the incident to return $userLastAssignedAnIncident } else { #If multiple with 0 incidents assigned randomly pick someone $randomUser = ($availableUser | Get-Random) $userFilter = "DistinguishedName -eq " + $randomUser.UserDN $userLastAssignedAnIncident = Get-SCClassInstance -Class (Get-SCClass -Name Microsoft.AD.User -ComputerName $SCSMManagementServer -Credential $smCreds) -Filter $userFilter -ComputerName $SCSMManagementServer -Credential $smCreds

            #return the user to assign the incident to return $userLastAssignedAnIncident }} elseif ($availableUser.Count -eq $null) { #If there is only one user with lowest number of active incidents, then get the user object from SCSM $userFilter = "DistinguishedName -eq " + $availableUser.UserDN $userLastAssignedAnIncident = Get-SCClassInstance -Class (Get-SCClass -Name Microsoft.AD.User -ComputerName $SCSMManagementServer -Credential $smCreds) -Filter $userFilter -ComputerName $SCSMManagementServer -Credential $smCreds

        #return the user to assign the incident to return $userLastAssignedAnIncident }

}

function populateHelpdeskUserList ($GroupName, $maxIncidentCount) { #Get the members of the AD Group #TODO: Currently set to Recursive. Remove -Recursive if direct membership is wanted $usersGroup = Get-ADGroup -Identity $GroupName | Get-ADGroupMember -Recursive

    #List to store analysts and number of active incidents $helpdeskUserList = @()

    #Calculate how many active Incidents assigned to each member foreach ($usr in $usersGroup) { $IRAssigned = 0#Find the user in SCSM $userFilter = "UserName -eq " + $usr.SamAccountName $userCI = Get-SCClassInstance -Class (Get-SCClass -Name System.Domain.User -ComputerName $SCSMManagementServer -Credential $smCreds) -Filter $userFilter -ComputerName $SCSMManagementServer -Credential $smCreds

        #Ensure the user is in SCSM if($userCI -ne $null) { #Get Active Incidents assigned to the user $IDs = (((Get-SCRelationshipInstance -TargetInstance ($userCI) -ComputerName $SCSMManagementServer -Credential $smCreds | ?{$_.RelationshipId -eq $WIAssigned.Id -and $_.IsDeleted -eq $false}).SourceObject | ?{$_.FullName -like "System.WorkItem.Incident*"}).Id)#if user has >0 incidents find count of how many active if ($IDs -ne $null) { $assignedIncidents = Get-SCClassInstance -Id $IDs -ComputerName $SCSMManagementServer -Credential $smCreds | ?{$_.Status.Name -eq "IncidentStatusEnum.Active"} $IRAssigned = $assignedIncidents.Count

                #If user not at capcity for incidents if ($IRAssigned -lt $maxIncidentCount) {#Get the list of Active Incidents assigned to user and DateTime of when last assigned $assignedIRToUser = @() foreach($IR in $assignedIncidents) { $assignedIR = New-Object PSObject -Property @{ Id = $IR.Id LastAssigned = (Get-SCRelationshipInstance -SourceInstance $IR -ComputerName $SCSMManagementServer -Credential $smCreds |?{$_.RelationshipId -eq $WIAssigned.Id -and $_.IsDeleted -eq $false}).LastModified } $assignedIRToUser = $assignedIRToUser + $assignedIR }

                        #Add the Incident details for this user to the list $helpdeskUser = New-Object PSObject -Property @{ UserName = $usr.SamAccountName UserDN = $usr.distinguishedName AssignedIncidents = $IRAssigned ListOfIncidents = ($assignedIRToUser | Sort-Object LastAssigned -Descending) } #Add user to list of available engineers $helpdeskUserList = $helpdeskUserList + $helpdeskUser } } else { #If the user has 0 incidents assigned #Add User to the list with 0 assigned $helpdeskUser = New-Object PSObject -Property @{ UserName = $usr.SamAccountName UserDN = $usr.distinguishedName AssignedIncidents = 0 ListOfIncidents = $null } #Add user to list of available engineers $helpdeskUserList = $helpdeskUserList + $helpdeskUser } } } return $helpdeskUserList }

try { Write-Output "Start Time: $(Get-Date -Format MM-dd-yyyy.HH:mm:ss)"

    #Get a list of all Active Incidents in the Support Group $activeIncidentsInSupportGroup = Get-SCClassInstance -Class (Get-SCClass -Name System.WorkItem.Incident -ComputerName $SCSMManagementServer -Credential $smCreds) -Filter "Status -eq IncidentStatusEnum.Active" -ComputerName $SCSMManagementServer -Credential $smCreds | ?{$_.TierQueue.DisplayName -eq $supportGroup} #Find assigned incidents, if there are any active incidents in support group if ($activeIncidentsInSupportGroup -ne $null) { $assignedIncidents = (Get-SCRelationshipInstance -SourceInstance ($activeIncidentsInSupportGroup) -ComputerName $SCSMManagementServer -Credential $smCreds | ?{$_.RelationshipId -eq $WIAssigned.Id -and $_.IsDeleted -eq $false}).SourceObject.Name #Get a list of unassigned Incidents in the Support Group $unassignedIncidents = $activeIncidentsInSupportGroup | where {$_.Id -notin $assignedIncidents} }

    Write-Output "Incidents to assign $($unassignedIncidents.Count)" #If there are Incidents to assign if($unassignedIncidents.Count -gt 0) {

        #Loop through every unassigned incidents starting with the oldest by created date foreach ($unassignedIncident in ($unassignedIncidents | Sort-Object CreatedDate)) {

            #Get the current list of users available to assign incidents to $TheUserList = populateHelpdeskUserList $ADGroupName $maxAssignedIncidents write-output $TheUserList

            #Get the user to assign this particular incident to $userToAssignIR = whoToAssignTo $TheUserList

            #Make sure there is a user in the list to assign to if ($userToAssignIR -ne $null){ New-SCRelationshipInstance -RelationshipClass $WIAssigned -Source $unassignedIncident -Target $userToAssignIR -ComputerName $SCSMManagementServer -Credential $smCreds write-output "ASSIGNED: $($unassignedIncident.Id) to: $($userToAssignIR.DisplayName)" } else { #No available users to assign the incident to write-output "FULL: No engineers available in $($supportGroup) to to assign incident $($unassignedIncident.Id)" if ($autoEscalate) { $unassignedIncident.TierQueue = $escalationGroup $unassignedIncident | Update-SCClassInstance write-output "ESCALATED: $($unassignedIncident.Id) to $($escalationGroup)" } elseif (!$autoEscalate) { Write-Output "Exiting script as assignment slots are full and auto escalation is disabled" break } } } } else { #No incidents needed to be assigned write-output "Currently no incidents to assign" } } catch { Write-Warning "ERROR: Something bad happened!" $_.Exception.Message }

 Install-Module AzureRM

 Import-Module AzureRM.OperationalInsights -MinimumVersion 1.0.8

 Login-AzureRmAccount

 Add-AzureDiagnosticsToLogAnalyticsUI

Adventure Question 1: Whatcha want to collect?

Adventure Question 2:

Adventure Question 3:

WHY

WHAT?

Get-AzureRmResourceProvider | Out-GridView

Get-AzureRmResourceProvider -ProviderNamespace Microsoft.Network | Out-GridView

Enabling Diagnostics via the Portal:

Enabling diagnostics via PowerShell:

Login-AzureRMAccount

#Find the Network Security Group by name:

$nsg = Find-AzureRmResource -ResourceNameContains "vmNSG" -ResourceType "Microsoft.Network/NetworkSecurityGroups"

#find the storage account you want to put the logging into

$stg = Find-AzureRmResource -ResourceNameContains "vmrg7033"

#enable diagnostic logging

Set-AzureRmDiagnosticSetting -ResourceId $nsg.ResourceId -StorageAccountId $stg.ResourceId -Enabled $true -RetentionInDays 1 -RetentionEnabled $true

#verify enabled is now True

Get-AzureRmDiagnosticSetting -ResourceId $nsg.ResourceId

There are already some great blogs out there, but in this series I wanted to take a simple look how this works, now it's generally available.

Part 1 link

Part 2 link

Triggering child Runbooks

In the previous blogs, we saw how to trigger a Runbook from an OMS Alert & read out the alert details.

Over the years, working with Automation tools, like System Center Orchestrator, I've become a fan of keeping my Runbooks simple!

Having big, monolithic Runbooks might look cool, but they can be a pain to troubleshoot & maintain. Splitting bigger tasks into small Runbooks also brings benefits, such as the ability to reuse!

In the spirit of this, the model I suggest is:

• Create a Trigger Runbook, which is triggered from the OMS Alert
• The trigger Runbook can parse the Alert RequestBody & then go and start another Runbook to complete a task

Why?

• If you've already got a Runbook which does something, you don't have to modify it to handle WebhookData input
• If you've got an existing PowerShell script, you can put it into Azure Automation, without having to alter the parameters

I also think it's cleaner to separate out the parsing of the alert data & the actual Remediation Runbook.

Step 1: Create a child Runbook

This is the Runbook which I want to be triggered by my existing OMSAlertTrigger Runbook.

I created a new Runbook called OMSAlertTriggerChild

I've just created a very basic Runbook, which takes 3 parameters.

Each parameter is simply written out. I saved & Published this Runbook.

Step 2: Call the child Runbook

Next, I've gone back to OMSAlertTrigger. I want to modify this, so it triggers the child Runbook instead of writing out data.

You can see below, I simply needed to call the Runbook OMSAlertTriggerChild by name & pass in the 3 parameters it accepts.

You can add it easy from selecting 'Runbooks' > Right click > Add to canvas:

You may have noticed I also added in -parallel so it can parse the alerts in parallel!

I then saved & published this Runbook

Step 3: Trigger the updated Runbook

As before, I'll trigger the Runbook by manually raising event 123 on one of my OMS managed machines. I ran this several times, to rest out multiple alerts:

Write-EventLog -LogName Application -Source MsiInstaller -EventId 123 -Message "hello"

I'll wait for a moment to get the Alert notification, then check the Runbook Job in the portal.

First thing you'll notice – you only see the Job for the OMSAlertTrigger, we don't see a separate job for the child Runbook:

However, if I look in the output for OMSAlertTrigger, I can see it shows me the output from the OMSAlertTriggerChild Runbook.

This is a lot simpler – I don't have to worry about linking job outputs together. I can simply see the output, although it called my child Runbook:

What about Hybrid Runbooks?!

In case you've not seen it, we can use the Hybrid Worker to run the Runbook in your datacenter. This is easy to setup, more info is at https://azure.microsoft.com/en-us/documentation/articles/automation-hybrid-runbook-worker/.

This is also exposed when we setup an Alert in OMS. In the screenshot you can see this time, I've selected to run on Hybrid Worker and selected my hybrid workers.

That's OK, but what about the Child Runbook? Although I've clearly indicated I wanted OMSAlertTrigger to run on my hybrid worker, where will OMSAlertTriggerChild be run? Obviously, I want this to be run on the hybrid worker also.

To test this out, I added a line to my OMSAlertTriggerChild Runbook, to write out the name of the machine it's running on:

Great News! It triggered my child Runbook on my Hybrid Worker too! J

As we can see in the screen below, the Parent Runbook Ran on prem. (see 'Ran on').

The Child runbook, which wrote the output said it's running on DC01, which is my on-prem hybrid worker.

To conclude

It sometimes makes sense to split up your Runbooks into bitesize pieces. It's often easier to manage, you may end up re-using parts!

If you trigger a Runbook on a Hybrid Worker & it calls another Runbook – that child Runbook is Run on the Hybrid worker also.

One final note – You can run an Azure Automation Runbook from PowerShell, using Start-AzureRmAutomationRunbook. This accepts the RunOn argument, which allows you to select it to Run on a Hybrid Worker. Let's say you wanted to trigger one Runbook from another, but have that run on a different Hybrid Worker, then that's how it could be done.

There are already some great blogs out there, but in this series I wanted to take a simple look how this works, now it's generally available.

Link to Part 1

Link to Part 3

Working with alert data

In the first blog, we saw how to trigger a Runbook from an OMS Alert.

In this blog, we'll look at the output data & how to extract useful information.

Step 1: Reading $WebhookData.RequestBody

In the last blog, we saw that all the useful info was in $WebhokData.Requestbody.

This looked a little cryptic, something like the below. Probably not what you want to read when sipping your morning coffee J

This is JSON data & we can easily parse this out in our Runbook. However, before we do that, let's just take a look at this info.

To read it manually, I did the following to make it easier:

• Copied the $WebhokData.Requestbody info from the Output screen
• Pasted it into a Visual Studio Code
• Saved as output.json
• Formatted the code (Alt Shift F) so it's easier to read!

Now that's easier to look at! I've highlighted some of the useful bits below:

• All of the content is inside SearchResults
• In Green, you can see there are 2 main sections:
  • __metadata shows us some metadata about the OMS search
  • value: This contains the actual event
• In Red you can see I've pulled out specifics, such as the Computer the event was on, the event log, the event ID & we can see the contents in the ParameterXml.

Step 2: Parsing the RequestBody

How do we read this JSON data & make use of it in our Runbook? Luckily PowerShell is to the rescue here!

This output is JSON formatted data, so all we need to do, is use the ConvertFrom-Json cmdlet.

Next up, I'm going to go back to my OMSAlertTrigger Runbook & click Edit.

First up, I will get delete $WebhookData.WebhookName and $WebhookData.RequestHeader, as I don't care about those right now.

Next, I'll use ConvertFrom-Json to load the RequestBody into a PowerShell object. This makes it much easier to navigate through the JSON.

For example, below we get the converted JSON and put it in $alertData

$alertData = ConvertFrom-Json $WebhookData.RequestBody

I can now navigate to the EventID using $alertData.SearchResults.value.EventID

Before I complete this, I've got two scenarios I want to cope with:

• What if the WebhookData is empty
• What if there are more than 1 alerts passed in the results (multiple value tags in the JSON)

Taking this into consideration, here's my new OMSAlertTrigger Runbook. A few notes:

• Line 5: We check the WebhookData isn't null. If it is, we write an error
• Line 6: Converts the JSON data into a PowerShell object
• Line 8: In case there are multiple alerts, we loop through each value in the SearchResults
• Line 9: I've selected a few elements from the JSON and written them out in a friendly way

Save & Publish the new Runbook!

Step 3: Trigger the updated Runbook

As before, I'll trigger the Runbook by manually raising event 123 on one of my OMS managed machines. I ran this several times, to rest out multiple alerts:

Write-EventLog -LogName Application -Source MsiInstaller -EventId 123 -Message "hello"

I'll wait for a moment to get the Alert notification, then check the Runbook Job in the portal.

Now if we look at the Job Output, we can see I raised the event 123 four times on the computer SQL.contoso.work.

The OMS Alert picked this up had occurred 4 times, so triggered the Runbook with al 4 alerts in the Web hook Data.

In the output, we can see that we parsed out each alert & wrote the output in a friendly way:

Great! So, now we know how to trigger the Runbook and read out the alert details in a straightforward way.

What other cool stuff can we do with this capability? That's what we'll look at in part 3

There are already some great blogs out there, but in this series I wanted to take a simple look how this works, now it's generally available.

This is part 1.

link to part 2

link to part 3

Triggering a simple Runbook

In this first blog we'll look at how to create a basic Runbook which is triggered by an OMS Alert.

The Runbook will simply write out the details of the alert.

Step 1: Create a Runbook

To start, let's create the Runbook which we want to be trigged by an alert.

Go to your Automation account & create a new Runbook. Here, I've created a PowerShell Workflow Runbook called OMSAlertTrigger:

This Runbook will take the alert information as the input. To do that, we need to create an input parameter called $WebhookData.

OMS will automatically handle passing the alert detail into the $WebhookData parameter.

This $WebhookData is documented at https://azure.microsoft.com/en-us/documentation/articles/automation-webhooks/ if you want to know more.

Essentially, it has 3 bits of information in its payload:

• WebhookName
• RequestHeader
• RequestBody

So we can understand what these 3 things are, this first Runbook is just going to write each of them as output.

In my Runbook, I entered the following PowerShell to write out the WebhookData info. Next, I clicked Save and then Publish.

NOTE: I have not manually created a Webhook on this Runbook. OMS will do this for me automatically.

Step 2: Create an on demand Alert

OK, now it's time to trigger the Runbook from an alert!

I've already configured OMS to collect the Application event logs from my Windows agents. Therefore, I know I can create an alert, based on anything in this event log.

In case you're not collecting Event Log data, it's configured from Settings > Data:

For this example, I want to be able to manually create an event, so I can test my Runbook on demand.

To do this, I'm using the Write-EventLog cmdlet in PowerShell.

Simply, go to any machine which is reporting to your OMS workspace, and run the following PowerShell:

Write-EventLog -LogName Application -Source MsiInstaller -EventId 123 -Message "hello"

Shortly after, I can see this in the OMS Log Search, by running the following query:

Type=Event EventID=123

And I can see the entries where I've created this event manually:

I want to setup an alert for this Event 123. So I click the Alert button:

Here's how I've configured my alert:

• You can see I'm checking every 5 minutes
• The Threshold is > 0, so this will fire if there are any 123 events
• I've configured this to send me an email
• Finally, you can see I've picked out to run my OMSAlertTrigger runbook

Click Save to activate this alert

Step 3: Trigger the Alert

OK! Now we're ready to see this in action.

I'll go to one of my OMS Agents and run the following PS to raise event 123 in the event log:

Write-EventLog -LogName Application -Source MsiInstaller -EventId 123 -Message "hello"

Shortly after, I got the email from my alert!

OK so hopefully this will have trigged our Runbook! To see this:

• Go to your Automation Account in the portal
• Click on the Jobs tile:

• You should be able to see the OMSAlertTrigger has run:

   

• Click this job to see the details

In the details of this Job, we can see it completed successfully.

Click Output to see what our Runbook Wrote out:

Remember, the Runbook wrote out the following:

    Write-Output $WebhookData.WebhookName

    Write-Output $WebhookData.RequestHeader

    Write-Output $WebhookData.RequestBody

That's what we can see in the screenshot above.

• WebhookName: Not that useful for us
• RequestHeader: Can see some information in there from the header. Again, not that useful
• RequestBody (in the red square): This is useful! This is all of the detail from the alert!

So you may be thinking that all that info looks a little cryptic! Don't worry, that's what we'll look at in Part 2…

If you're using SMA, you've probably noticed there are many modules imported for you out-of-the-box. Among others, this includes modules for some of the System Center components: Configuration Manager, Data Protection Manager, Operations Manager, Virtual Machine Manager & SPF.

However, in SMA, there is no module for Service Manager by default. This isn't a big deal, as your PowerShell workflows can still run Service Manager cmdlets, as long as you install the Service Manager console (which includes the PowerShell module) on the Runbook Worker.

The benefit of having the PowerShell module imported into SMA or Azure Automation as an asset is simple – the Service Manager cmdlets are shown alongside all of the other activities in the UI. The module is used only to make the authoring experience easier, giving benefits such as storing a Service Manager credential as an asset in SMA or Azure Automation.

One thing to note – this is NOT the whole Service Manager PowerShell module, but the portable module. This basically means that we define what cmdlets are available in the module & store credentials, but don't actually copy any of the resources over. Remember, this is just used to show the activities in the authoring experience. There's more information about Portable Modules at http://blogs.technet.com/b/orchestrator/archive/2013/11/04/service-management-automation-portable-modules-what-why-and-how.aspx.

I created the portable module, using the New-SmaPortableModule command, which is part of the SMA PowerShell module. I've attached it to this blog post, so you can simply import it, rather than having to roll your own. This was created in a Service Manager 2012 R2 UR5 environment.

Importing the Module in SMA

To import the Service Manager Module via the Azure Pack site, go to Automation > Assets > Import Module. You can simply upload the .zip file attached to this blog.

Importing the Module in Azure Automation

From the Azure Automation blade, you can go to Assets > Modules > Add a Module. Upload the zip file attached to this blog.

The module takes several minutes to be extracted and imported. After a few minutes, you'll see the Service Manager activities available to use in your Runbooks.

System.Center.Service.Manager-Portable.zip

First up – How MANY users are connected?

Before we look at how to see who is connected to a Management Server, I wanted to look at how to simply see how many people are connected. The easiest way to do this is from perfmon on the Management Server.

Using the counter OpsMgr SDK Service\Client Connections Using Cache. This will give you read of connections to the Data Access Service in real time.

OPTION 1 – Using the SCOM PowerShell Module

Operations Manager has a nifty & simple way of getting the currently connected users via PowerShell. Unfortunately, this same functionality isn't available in the Service Manager PowerShell interface. However, due to the shared infrastructure in System Center, we can actually use the Operations Manager shell to query Service Manager.

Whenever you install the Operations Manager console, you'll also get the "Operations Manager Shell". This will open up the Operations Manager PowerShell module and connect to your Operations Manager Management Server.

First, we can disconnect from Operations Manager, then point it at a Service Manager Management Server instead. We can then simply query for the list of connected users:

Get-SCOMManagementGroupConnection | Remove-SCOMManagementGroupConnection

New-SCOMManagementGroupConnection -ComputerName SERVICEMANAGERSERVER

(Get-SCOMManagementGroup).GetConnectedUserNames()

This will give us the list of the currently logged on users to a Service Manager Management Server:

OPTION 2 – Using .Net

Alternatively we can use the .Net interface into Service Manager. This is an updated version of what's been previously posted at http://blogs.technet.com/b/servicemanager/archive/2010/02/01/getting-a-list-of-connected-users.aspx, as many folks had a problem running it.

Here we're using the GetConnectedUserNames method on the EnterpriseManagementGroup class, as per http://msdn.microsoft.com/en-us/library/gg460200.aspx.

EnterpriseManagementGroup emg = new EnterpriseManagementGroup(managementServer);

foreach (String userName in emg.GetConnectedUserNames())

{

Console.WriteLine(userName.ToString());

}

I've attached a Visual Studio project, which includes the SMConnections.exe which is ready to run.

You can run: "SMConnections.exe" from the command line, which will default to localhost to run on the Management Server. Alternatively you can run "SMConnections.exe SERVERNAME" to remotely connect to a Management Server.

OPTION 3 – Using Orchestrator

Another way we can do this is in Orchestrator, using the 'Run .Net Script' activity.

In the 'Initialize Data' activity, I've created 1 string input, called managementServer.

In 'Run .Net Script', I set the language to C# and input the following:

Note, I've looped through the users, adding each to a variable called userNameList, separating each with a comma and space.

On the advanced tab, I've added in the reference & Namespaces. You'll have to point to the Service Manager SDK binary, which is available at C:\Program Files\Microsoft System Center 2012 R2\Service Manager\SDK Binaries\Microsoft.EnterpriseManagement.Core.dll. We're using the namespaces 'System' and 'Microsoft.EnterpriseManagement'.

Finally, I've published out the userNameList variable from the code:

The final activity can write the output however you like. I used a 'Send platform event', but it'd be a good idea to write to a log, DB, etc.

SMConnections.zip

For example, the 'Incident Classification' list can be altered to include 'Facilities Problems' or 'Mobile Device Problems'.

I've had a few questions recently, asking where the customizations to your lists are stored. This blog will explain how to find that out.

#1 – Where is the List defined?

Let's look at the Incident Lists as an example.  From the Lists, we can see which Management Pack these are stored in:

We can see that this is a Sealed Management Pack:

If you export the Incident Management Library (System.WorkItem.Incident.Library.xml) file, this is what you'll see:

        <EnumerationValue ID="IncidentTierQueuesEnum" Accessibility="Public" />

        <EnumerationValue ID="IncidentClassificationEnum" Accessibility="Public" />

        <EnumerationValue ID="IncidentResolutionCategoryEnum" Accessibility="Public" />

This MP defines that there are some lists, such as the Incident Classification. HOWEVER there is no contents of the list in this Management Pack.  Basically, this MP has created some lists, but not populated them with anything.

#2 – Where is the List contents?

In this case, we know that out of the box, the Incident Classification list does contain some items.  So where are they? There are a few ways to do this (look through the MP XML, PowerShell), but I'm using SQL here

The query below can be run against the ServiceManager database. All you need to do is replace Incident Classification with the display name of the List from the Service Manager console. For example, you could replace with Service Request Support Group.

select EnumTypeName, lt.LTValue, mp.MPName, mp.MPFriendlyName

from enumtype et

JOIN LocalizedText lt on

lt.ElementName = et.EnumTypeName

JOIN ManagementPack mp on

mp.ManagementPackId = et.ManagementPackId

where lt.LanguageCode = 'ENU'

and lt.LTStringType = 1

and et.ParentEnumTypeId = (

select et.EnumTypeId from LocalizedText lt

JOIN EnumType et on

(et.EnumTypeName = lt.elementname)

WHERE lt.LTValue = 'Incident Classification'

AND lt.LTStringType = 1

AND lt.LanguageCode = 'ENU')

and et.Enabled = 1

order by et.Ordinal

In the output we can see all of the items in the list, their display name and the Name of the MP they are stored in:

Note – the ones with the EnumTypeName which are Enum.{GUID} are items generated via the Service Manager console.  The others are out-of-the-box.

Here, we can see that the list Items are stored in the Service Manager Incident Management Configuration Library (ServiceManager.IncidentManagement.Configuration.xml) MP. Of course, as we're updating this in the console, the MP is unsealed:

#3 – Why are they in that MP?

You may notice, when you edit the Incident Classification List, it just works. I.e. It doesn't ask you where you want to store you custom list items.

The reason is:

• The List is defined in the SEALED MP we identified in step 1
• Out-of-the-box, that list has got items added to it.  They have already been stored in UNSEALED MP from step 2.
• Therefore, all future additions to the list will go in the same MP as step 2.  You've no control over that from the UI.

You may notice that sometimes when you edit a list, you get a pop-up, asking which MP to store the list contents in:

This happens for 1 of 2 reasons:

• The List was defined in a sealed MP & hasn't been extended yet.  Therefore you have to select the unsealed MP to use.  This is a one-time thing, then all items will go in the same unsealed MP.
• The list may already have items in it, but they are inside of a sealed MP.  Therefore, your extensions will need to be in an unsealed MP.  Again, this is a one-time selection.