System Center 2012 Operations Manager Web Application Monitoring Example
While publishing “Microsoft Office 365 Administration Inside Out”, we realized we had illustrated a fairly well-understood way to monitor your O365 tenant with System Center 2012 Operations Manager (SCOM), which may produce false positives. In “Microsoft Office 365 Administration Inside Out”, chapter 5 instructs the reader to browse to this web page and download the Office 365 management pack (MP). This MP is merely used as an example of how to use the System Center 2012 Operations Manager (SCOM) Web Application Availability Monitoring wizard to create a management pack of a few synthetic transactions, aka as fake or test transactions.
Download the following MP and Import into your System Center 2012 Operations Manager infrastructure. You will need to edit the MP as described in Chapter 5 and 6 of “Microsoft Office 365 Administration Inside Out”.
It is important to understand that these synthetic transactions only monitor specific entities and relying on them to determine the state of your O365 tenant can lead to false positives. Let’s use Microsoft Exchange as an example. Most Microsoft Exchange servers store mailboxes in more than one mail database, known as the EDB file. There are a variety of reasons for administrators doing this, but the point here is that monitoring a mailbox’s availability only reflects its mail database availability. In O365, mailboxes may be spread across a number of mail databases. This means we may be incorrectly stating that email is running, while only some mailboxes are available.
Microsoft Office 365 distributes mailboxes across numerous mail databases, abstracting this application layer from the user and customers. Monitoring this O365 mailbox will only validate that the mail database it resides on is available. This begs the question; how do we monitor O365 more accurately?
Office 365 has a component called the System Health Dashboard (SHD), which is a matrix of health states of various O365 workloads over the last 7 days. As you can see, O365 has 8 Exchange Online workloads that are exposed in this dashboard and need to be monitored along numerous others services and workloads.
O365 provides the following 7 services and 35 workloads:
- Exchange Online – Email and Calendar access, E-Mail timely delivery, FOPE Administration, FOPE Encryption, FOPE Quarantine, Management and Provisioning, Sign-In and Voice mail
- Identity Service – Administration and Sign-In
- Lync Online – All Features, Audio and Video, Dial-In Conferencing, Federation, Instant Messaging, Management and Provisioning, Mobility, Online Meetings, Presence, Sign-In
- Office 365 Portal – Administration and Portal
- Office Subscription – Licensing and Renewal, Network Availability and Office Professional Plus Download
- Rights Management Service
- SharePoint Online – Access Services, Custom Solutions and Workflows, InfoPath Online, Office Web Apps, Project Online, Provisioning, Search, SharePoint Features, SP Designer and Tenant Admin
The O365 Service Health Dashboard (SHD) is the authoritative source for monitoring an O365 tenant. This means we need to query the SHD with SCOM to determine the state of the services and workloads we are interested in alerting on.
The concepts behind this entails 3 different parts:
- Run a script from a SCOM Watcher Node that queries the SHD
- Have the script login to the O365 Admin Portal (https://login.microsoftonline.com)
- Have the script read the servicestatus.aspx page and determine which one of the 9 values exists for the workload being monitored
- Have the script write the appropriate event log entries and update monitor status
- O365 custom MP rules pick up event entries and generate alerts accordingly
We have been leveraging PowerShell and have made progress, but not successfully logged-in due to, what we identified as a Java component requirement.
=================================
$r = Invoke-WebRequest https://portal.microsoftonline.com/ServiceStatus/ServiceStatus.aspx -SessionVariable O365
$O365
$form = $r.Forms[0]
$form | Format-List
$form.Fields["cred_userid_inputtext"] = "username@yourdomain.onmicrosoft.com"
$form.Fields["cred_password_inputtext"] = "password"
$r = Invoke-WebRequest -Uri ("https://portal.microsoftonline.com/ServiceStatus/ServiceStatus.aspx" + $form.Action) -WebSession $O365 -Method POST -Body $form.Fields
$r.StatusDescription
==================================
Next we need to query the E-Mail and calendar access section of the aspx page. From this section of the page, we can retrieve the current state of the workload.
We are retrieving the first column because it is the only state we need. O365 keeps 30 days of history for the tenant service monitoring, but SCOM can maintain months to years’ of data.
As of this posting, we have completed the following PowerShell commands to accomplish this. This script is a work in progress because we are at an impasse attempting to move beyond the login page.
==================================
$EMailServiceName="E-Mail and calendar access"
$HTML = gc '.\Service health.htm'
$section=$false
$result=""
foreach($line in $HTML)
{
if($line -match "ServiceNames" -and $line.Contains($EMailServiceName)){$Section=$true}
if($section -and $line.Contains("TodayCol") -and $line -match "title=`"([^`"]*)`""){$result=$matches[1]}
if($line -match "SubServiceNames" -and -not $line.Contains($EMailServiceName)){$Section=$false}
}
write-host "Service:" "'"$EMailServiceName"'" "Curent Status is:" $result
Finally, once the value is retrieved, we can write to the Application Event Log like this:
write-eventlog -logname Application -source O365MP -eventID 301 -entrytype Information -message "O365 message" -category 1 -rawdata 10,20
==================================
There are 9 states an O365 Service/workload can be in. This table identifies them and how we could reflect them in SCOM:
SHD Service State |
SHD Service State Icon |
SCOM Monitor Status |
SCOM Alert |
SCOM Alert Status |
Normal service |
|
Green |
No |
Healthy |
Service Restored |
|
Green |
No |
Healthy |
Service degradation |
|
Yellow |
Yes |
Warning |
Restoring service |
|
Yellow |
Yes |
Warning |
Extended recovery |
|
Yellow |
Yes |
Warning |
Service interruption |
|
Red |
Yes |
Error |
Additional information |
|
Green |
No |
Informational |
PIR published |
|
Green |
No |
Informational |
Investigating |
|
Green |
No |
Informational |
There are many pages online that guide you through creating a monitor in SCOM. Here are a few we recommend. While they may be written for SCOM 2007, they are applicable for SCOM 2012.
How to create a monitor based on a script
Create a Script-Based Unit Monitor in OpsMgr2007 via the GUI
Creating a SCOM monitor for an average value
Once this process is complete and working successfully, select a different workload to retrieve the state and repeat for each workload you are interested in monitoring.
This posting is provided "AS IS" with no warranties, and confers no rights. Use of included utilities is subject to the terms specified at https://www.microsoft.com/info/cpyright.htm