In this post I am going to walk you through how to do achieve this when you have put all your RunAs Accounts and Groups into a .CSV file. 
The runbook will run 2 times a day to verify that you have up to date membership/distribution and don't have to have a scheduled task or anything else running.

THE OVERALL RUNBOOK

1. START TIME ACTIVITY
First we need to build our Monitor Date Time activity.  I want this to run 12 hours so it will look like this(you can run this to your own desires to meet your SLA):

2. VERIFY FILE EXISTANCE OF .CSV FILE
We need to create our .CSV on the RB Server(make sure it is on the server running the runbook and not the designer) :) It will look like this and you're always free to change it up but make sure you change the PowerShell that goes along with this!
Again, I created mine in the C:\SCOM Folder so make sure to update this as well when you do yours.


Once this has been created we need to do a validation in the runbook to make sure that the file exists:


2.1 POTENTIAL ERROR HANDLING
Here you can see that I just added different ways of alerting or notifying that file didn't exist:

3. GET DATA FROM CSV
Next we need to import the .CSV Data and break it into useable data for SCO data bus:


SCRIPT:
$FilePath= 'UPDATE TO POINT TO NAME AND PATH OF FILE FROM PREVIOUS ACTIVITY'
$Data = Import-CSV $FilePath
$Array = @()
ForEach ($Row in $Data)
{
   $AccountName= $Row.'Account'
   $Group = $Row.'SCOMGroup'
    $str = $AccountName +","+ $Group
    $Array += $str+";"
}

PUBLISHED DATA:
Array -> String -> Array


4. DEPLOY/VALIDATE ACCOUNTS
Once we have this we can actually do the account Distribution by breaking it into RunAs Account and associated SCOM Group you wish to use ( script from Kevin's blog modified slightly to pass in the data).


SCRIPT:
Import-Module OperationsManager
New-SCOMManagementGroupConnection YOUR MGMT SERVER

#Get all the health service class instances in the management group
$HealthServiceClass = Get-SCOMclass -DisplayName "Health Service"| Get-SCOMClassInstance

$Details = 'UPDATE TO ARRAY FROM PREVIOUS ACTIVITY'
$Info = $Details.Split(';')
foreach ($Account in $Info)
{
    If (![string]::IsNullOrEmpty($Account))
    {
        $Account = $Account.Trim()       
        $split = $Account.split(',')
        $RunAsDisplayName = $split[0].Trim()
        $DistGroupName = $split[1].Trim()               

        #Log an event that our script is starting
        Write-EventLog –LogName 'Operations Manager' –Source “Health Service Script” –EntryType Information –EventID 3250  –Message “RunAs HealthService Distribution Script Starting for account: ($RunAsDisplayName) and group: ($DistGroupName)”

        #Get the SCOM Run As account by display name
        $RunAs = Get-SCOMRunAsAccount $RunAsDisplayName

        #Choose a group by displayname. 
        #This group contains Windows Computers objects for agent managed computers and for cluster virtual server names.
        $DistComputerNames = Get-SCOMGroup -DisplayName $DistGroupName | Get-SCOMClassInstance | Select-Object -Property DisplayName

        #Exit script and log error if group returns no valid instances of Windows Computer property displayname
        IF ($DistComputerNames -eq $null)
        {
            #Log an event that our script is ending in error
            Write-EventLog –LogName 'Operations Manager' –Source “Health Service Script” –EntryType Warning –EventID 3252  –Message “RunAs HealthService Distribution Script ended in error.  The group ($DistGroupName) was not found or contained no objects!”
        }
ELSE
        {
            #Set DistAgents to empty
            $DistAgents = @()

            #Get an array of Health Service Objects which match the displayname of the group membership objects
            Foreach ($DistComputerName in $DistComputerNames)
            {
                $DistAgents += $HealthServiceClass | where {$_.DisplayName -eq $DistComputerName.DisplayName}
            }

            IF ($DistAgents -ne $null)
            {
                #Compare lists (diff) to get a list of all Computers in the group that do not have a matching HealthService. 
                #Assume these are Network Name values for clustered resource groups
                $DistClusters = Compare-Object $DistComputerNames $DistAgents -Property DisplayName -PassThru
            }
            ELSE
            {
                #Assume that the group only contained Windows Cluster objects and no agents.
                $DistClusters = $DistComputerNames
            }

            # If there are no clusters in the group skip the cluster node section of the script
            IF ($DistClusters.count -ge 1)
            {
                #Set UknownObject array equal to null to ensure it is empty from any previous script runs
                $UnknownObjects = @()

                #Get the relationship ID we need to find nodes that host a cluster
                $rid = Get-SCOMRelationship -DisplayName 'Health Service manages Entity'

                #Get all Cluster virtual server class instances in an array
                $vsclass = Get-SCOMClass -name 'Microsoft.Windows.Cluster.VirtualServer' | Get-scomclassinstance

                #Create a loop to find the node names of each cluster and add those names to an array
                foreach ($clname in $DistClusters)
                {
                    #Get the Virtual Server class instance for each cluster name
                    $vs = $vsclass | where-object {$_.DisplayName -eq $clname}

                    #Continue with the script if we got a match for a virtual server class
                    IF ($vs -ne $null)
                    {
                        #Get the nodes in an array which have a health service relationship managing the virtual server cluster name
                        $nodes = Get-SCOMRelationshipInstance -TargetInstance $vs | Where-Object {$_.relationshipid -eq $rid.id} | Select-Object -Property SourceObject

                        #Get an array of SCOM Agent objects which match the displayname of objects in the nodenames array and check first to ensure no duplicates
                        Foreach ($node in $nodes)
                        {
                            If ($DistAgents -notcontains $node.SourceObject)
                            {
                                $DistAgents += $HealthServiceClass | where {$_.DisplayName -eq $node.SourceObject}
                            }
                        }
                    }
                    #If objects in the group do not match the displayname for a virtual server class nor a healthservice displayname add them to unknown objects array
                    ELSE
                    {
                        $UnknownObjects += $clname
                    }  
                }
            }

            #Get the current RunAs account distribution as it exists today and save it in an array
            #Comment out this entire section if you want to ignore the current distribution and ONLY go with what it in the group
            $RunAsDistOld = (Get-SCOMRunAsDistribution $RunAs).securedistribution

            #Check and see if there is at least one health service distribution defined - ignore if none found
            #If at least one distribution is found, check for duplicates then add non dupes to the DistAgents array
            IF ($RunAsDistOld.count -ge 1)
            {
                Foreach ($RunAsOld in $RunAsDistOld)
                {
                    IF ($DistAgents.DisplayName -notcontains $RunAsOld.DisplayName)
                    {
                        $DistAgents += $RunAsOld
                    }
                }
            }

            $DistAgents = $DistAgents | sort-object -Property DisplayName -Unique

            #Set (replace) the RunAs distribution list for the defined RunAs account to whatever we found in DistAgents
            Set-SCOMRunAsDistribution ($RunAs) -MoreSecure -SecureDistribution $DistAgents

            #Log event that script is complete, additional variables for logging purposes
            $RunAsName = $RunAs.Name
            $DistAgentsCount = $DistAgents.count
            $RunAsDistOldCount = $RunAsDistOld.count
            If ($UnknownObjects.Count -lt 1)
            {
                $UnknownObjects = 'NONE'
            }

            Write-EventLog –LogName 'Operations Manager' –Source “Health Service Script” –EntryType Information –EventID 3251  –Message “RunAs HealthService Distribution Script Complete. Account: $RunAsName, Configured $DistAgentscount Healthservice objects.  Previously there were $RunAsDistOldcount objects configured.  Unknown Objects identified: $UnknownObjects.  Health service objects added: $DistAgents”
        }
    }
}

PUBLISHED DATA(Now this is not necessary, I just put this here so SCO can show me what is being added/exists):
DistAgents -> String -> DistAgents
RusAsDistOld -> String -> RunAsDistOld

NOW RENAME YOUR RUNBOOK AND OFF YOU GO!!!!

HERE IS SOME OF THE OUTPUT FROM MINE:
You can see that I have removed the SCORCH server from my Orchestrator Account and the RB will automatically added it back in for me.
Removed:



SCO Added:


Runbook executes and output....you can see my output because I have logging turned on (ONLY RECOMMENDED FOR TROUBLESHOOTING PURPOSES AND INITIAL BUILD OF RB....DO NOT LEAVE TURNED ON!)


After execution in SCO RB Server OpsMgr Event Log:


When I have a BAD group it will stop on the Bad Group and log.  Now this will be on the server that the runbook is executing(Just keep that in mind when troubleshooting):

Disclaimer:

This example is provided “AS IS” with no warranty expressed or implied. Run at your own risk. The opinions and views expressed in this blog are those of the author and do not necessarily state or reflect those of Microsoft.

**Always test in your lab first**  Do this at your own risk!! The author will not be held responsible for any damage you incur when making these changes!

Account.csv

First thing is we do get our alert, but this is what our runbook will key off of.


The reason for the above alert is because our SQL Monitoring Account Distribution List look like this without SERVICEMANAGER.Lab.com:

Then from this we are able to set up our runbook to take care of the rest.

1. GET MISSING CREDS ALERT
Set up the Monitor alert activity as below: 
Name = System Center Management Health Service Credentials Not Found Alert Message
Now I use an Updater to forward my alerts so the bottom part is not necessary for you unless you don't want it to go to ticket but still get picked up by SCO.

From this activity we will pick up any alerts that match the criteria like the one in the first screenshot once the resolution state is updated to Processed by SCORCH. 

2. GET SSID:
Next we use the alert description to break apart the data and pull back the important piece for this to work: Account SSID. 


In our runbook we will utilize the Run .NET Script activity for the next 3 activities and each will use PowerShell to execute the scripts below:


SCRIPT:
$string = 'UPDATE TO POINT TO DESCRIPTION FROM PREVIOUS ACTIVITY'
$string = $string.split([environment]::NewLine)
$s= $string | where{$_.contains('SSID:')}
$SSID = $s.Split(':')[1].Trim()

PUBLISHED DATA:
SSID -> String -> SSID



3. GET THE ACCOUNT
Next we need to get the actual account we need to add the computer to:

Assistance from Michiel Wouters blog on how to decipher the SSID!
SCRIPT:
#Import OM Module
Import-Module OperationsManager;
New-SCOMManagementGroupConnection YOUR MGMT SERVER


$SSID = 'CHANGE TO POINT TO SSID FROM PREVIOUS ACTIVITY'
Get-SCOMRunAsAccount | Sort Name | % {$string = $null;$_.SecureStorageId | % {
 $string = $string + "{0:X2}" -f $_}

[string]$RunAsAccountSSID = $string
if ($SSID -like $RunAsAccountSSID)
{$RunAsAccountName = $_.Name}
}

PUBLISHED DATA:
RunAsAccountName -> String _> RunAsAccountName



4. ADD TO DISTRO LIST
From here we pass along the RunAs Account and get his current list if any and then we add the original alerting machine to the Account:


SCRIPT:
Import-Module OperationsManager;
New-SCOMManagementGroupConnection YOUR MGMT SERVER


$RAAccount = Get-SCOMRunAsAccount -name 'UPDATE TO POINT TO RUNASACCOUNT FROM PREVIOUS ACTIVITY'
$monitoringObjectId = get-SCOMMonitoringObject -Id 'UPDATE TO POINT TO MONITORINGOBJECTID FROM MISSING HEALTH CRED ACTIVITY'
$Group=  $RAAccount.ManagementGroup
$connection = [Microsoft.SystemCenter.OperationsManagerV10.Commands.OMV10Utility]::GetConnectionForManagementGroup($Group)
$hsClass = $group.EntityTypes.GetClasses("Name = 'Microsoft.SystemCenter.Healthservice'") |%{ $_ }
$poolClass = $group.EntityTypes.GetClasses("Name = 'Microsoft.SystemCenter.ManagementServicePool'") |%{ $_ }
[Microsoft.EnterpriseManagement.Monitoring.MonitoringObject[]] $distributionList = @()
if (((Get-SCOMRunAsDistribution $RAAccount).securedistribution).count -ge "1")
    {$distributionList  += (Get-SCOMRunAsDistribution $RAAccount).securedistribution
    }
foreach($o in $monitoringObjectId)
   {
     if($destination = ($o -as [Microsoft.EnterpriseManagement.Administration.AgentManagedComputer]))
        {
          $distributionList += $destination.HostedHealthservice
        }
     elseif($destination = ($o -as [Microsoft.EnterpriseManagement.Administration.ManagementServer]))
        {
          $distributionList += $destination.HostedHealthservice
        }
     elseif($destination = ($o -as [Microsoft.EnterpriseManagement.Administration.ManagementServicePool]))
        {
          $distributionList += ( Get-SCOMClassInstance -Id $destination.Id -ScSession $connection )
        }
     elseif(($destination = ($o -as [Microsoft.EnterpriseManagement.Monitoring.MonitoringObject])) -and ($destination.IsInstanceOf($hsClass) -or $destination.IsInstanceOf($poolClass)))
        {
          $distributionList += $destination
        }
   [Microsoft.SystemCenter.OperationsManagerV10.Commands.OMV10Utility]::ApproveRunasAccountForDistribution($Group, $RAAccount, $distributionList)
   }


Once you have this setup and the runbook fires on the Alert you will see that once it is complete you will see all 4 steps as Green 


The Missing credentials alert is now closed by System due to account being distributed:

New Updated Distribution list for SQL Monitoring Account:

Disclaimer:

This example is provided “AS IS” with no warranty expressed or implied. Run at your own risk. The opinions and views expressed in this blog are those of the author and do not necessarily state or reflect those of Microsoft.

**Always test in your lab first**  Do this at your own risk!! The author will not be held responsible for any damage you incur when making these changes!

I have tried in other posts to get this to work but I am sure as many of you know when you try to restart the service from a task it stops and never restarts.  This became a problem when I was trying to automate changing Management Groups between servers, etc. 

The issue is that you have to create a breakaway job from the task to allow it to one restart the service but 2. give you back data.  So in working with a customer who had to do a lot of work on his DCs and restart their services and of course many of us don't have access to those servers.  Well we do know from the from some of the built in monitors that they have recovery tasks to restart the service when they pass X threshold of Handle Count. So he actually broke out the script to get the Java script portion and I worked on getting it converted to a VBscript. 

Now you use this to do almost anything!

SCRIPT:

dim objWMIService, objProcess, strCommand
dim oShell, oShellEnv, strShell, computerName, oAPI

Set oShell = WScript.CreateObject("WScript.Shell")
Set oAPI = CreateObject("MOM.ScriptAPI")

set oShellEnv = oShell.Environment("Process")
computerName = oShellEnv("ComputerName")

strCommand = "cmd /c net stop HealthService & cmd /c net start HealthService"

Set objWMIService = GetObject("winmgmts:{impersonationLevel=impersonate}!\\.\root\cimv2")

Call oAPI.LogScriptEvent("SCOMTaskAgentRestart.vbs", 1001, 4, "Attempting restart of SCOM Health Service on " & computerName & " initiated by user in console")

Set objProcess = objWMIService.Get("Win32_Process")
Set objProgram = objProcess.Methods_( _
"Create").InParameters.SpawnInstance_
objProgram.CommandLine = strCommand

Set strShell = objWMIService.ExecMethod( _
"Win32_Process", "Create", objProgram)

WScript.echo "Restarted SCOM Health Service on " & computerName

To get this to work I created a new task and targeted the Windows Server class

and pasted the script from above with no parameters:

Now when I save and let it get pushed to all machines I can see it when I look in the windows computer view

Now to test I just select a machine and select the task

Output from the executed task. Yes, what you are assuming is correct.  It will always show that it is successful, but the difference here is that the task actually completes and doesn't just spin never knowing if it actually finished. 
If it were to fail you would receive a Heartbeat issue(just something to keep in the back of your mind) :



From the Task Status View in the console:


Now we go to the server to verify!  We are looking for a 103, 102, and 105 event at 5:36:



In later posts I will actually tie this into other tasks that I use for automation and call from Orchestrator!


Also, please feel free to leave ideas on how you think you can use this and tying it into other tasks to make the lives of us SCOM admins easier! :)

Disclaimer:

This example is provided “AS IS” with no warranty expressed or implied. Run at your own risk. The opinions and views expressed in this blog are those of the author and do not necessarily state or reflect those of Microsoft.

**Always test in your lab first**  Do this at your own risk!! The author will not be held responsible for any damage you incur when making these changes!

• The "Remove-DisabledClassInstance" Windows PowerShell command times out without completing
  
  The cmdlet takes lots of time to complete in large SCOM installations and sometimes times out without completing its work. This is fixed by optimizing the query that is used for deletion. 

• Duplicate closed alerts
  
  The Find option in a message view sometimes gives results of the current search together with the results from previous searches. This occurs because of an update issue in the grid and is fixed so that values are displayed that correspond only to the current search. 

• Topology widget objects lose location when they are opened in a console that has a different locale and decimal format
  
  Topology widgets that are created in one locale but are viewed from another locale display an incorrect location because of the widget's inability to interpret locale-specific formatting. This issue is fixed, and users can now create and view topology widgets from different locales.

• WebConsole Details widget does not display anything
  
  When you click a message in the Alert widget, the details of the message are not displayed in the details widget on WebConsole. This issue is fixed by fixing the XAML page of the details widget. The XAML page was preventing the data from being displayed on the page.

• Top 10 Performance widgets (WebConsole) are sometimes empty
  
  The Top 10 Performance widgets (WebConsole) are sometimes empty because of a performance issue. This issue was fixed in Update Rollup 3 through an MPB file that requires a manual import. Because many users missed importing the MPB, a fix is included in this update rollup as an SQL query update.

• Problem with decoding SCOM trace log files
  
  Sometimes Operations Manager trace logs do not decode but generate the error "Unknown(X ): GUID=XXX (No Format Information found)." This occurs because the latest TMF files that contain formatting information for the related traces are missing. All TMF files are updated to make sure that traces are formatted correctly.

  Unix/Linux Management Pack

• JEE: WebLogic 12.1.3 servers on Linux or Solaris are not discovered
  
  WebLogic 12cR1 Rel3 Java Application Servers that are installed on Linux and Solaris are not discovered.
  
  Note The UNIX or Linux agent must be updated after you install the Management Pack update to resolve this issue.
  
  Relevant log messages:
  
  /var/opt/microsoft/scx/scx.log

  2015-02-04T15:28:18,797Z
  Trace [scx.core.common.pal.system.appserver.appserverenumeration:379:15739:139715041470400] Weblogic process does not contain the 'platform.home' or 'weblogic.system.BootIdentityFile' commandline argument.
  
  2015-02-04T15:28:18,797Z
  Trace [scx.core.common.pal.system.appserver.appserverenumeration:462:15739:139715041470400] Merging previously known instances with current running processes

   

• In rare cases, lots of omiagent processes may be observed on a UNIX or Linux computer
  
  If a single instance of data that is returned by an OMI provider is larger than 64 kilobytes (KB), the omiagent process becomes unresponsive, and additional omiagent processes are created. This issue can occur on a managed UNIX- or Linux-based computer if a rule or monitor returns an instance of data larger than 64 KB. This is most likely to be seen when a custom shell command or script rules return lots of data in StdOut.
  
  Note After you upgrade the Operations Manager UNIX/Linux agent to the version that is contained in this Management Pack update, an instance that exceeds 64 KB will be truncated and will generate the following message in the scx.log file. You can use the log message to identify and change the workflow that is returning too much data.

  ExecuteCommand: Exceeded maximum output size for provider (64k), output truncated. Monitoring will not be reliable! Command executed: "

   

• The SSLv3 protocol cannot be disabled in OpenSSL as used by the UNIX and Linux agents
  
  With this update, you can now disable the SSLv3 protocol for the Operations Manager UNIX and Linux agents. You can do this by adding a property to omiserver.conf (/etc/opt/microsoft/scx/conf). If SSLv3 is disabled, the UNIX and Linux agent will reject connections that cannot be negotiated by using TLS encryption. If SSLv3 is enabled (this is the default configuration), SSLv3 will be used only if TLS encryption cannot be negotiated.
  
  omiserver.conf options for configuration SSLv3

  Collapse this tableExpand this table

https://support.microsoft.com/en-us/kb/3051169 

Issues that are fixed in this update rollup

Issue 1
It takes many minutes for the Microsoft.Windows.CheckNTServiceStateMonitorType function to determine that the service is down.

Issue 2
An attempt to apply sensitivity rules fails, and you receive the following error message:

Issue 3
GSM locations are geotagged incorrectly on Map Widget.

Issue 4
The MonitoringHost.exe process issues too many calls to System Center Failover Clustering.

Issue 5
The re-synchronization of all managed entities from Operations Manager to the Operations Manager Data Warehouse database causes duplicate entries, and an exception that resembles the following is thrown:

Issue 6
If data is truncated and is in Terminal status to a workflow, event ID 31551 is logged, and an exception that resembles the following is thrown:
Failed to store data in the Data Warehouse. The operation will be retried.
Exception 'InvalidOperationException': The given value of type String from the data source cannot be converted to type nvarchar of the specified target column.
One or more workflows were affected by this.
Workflow name: Microsoft.SystemCenter.DataWarehouse.CollectAlertData

Issue 7
An arithmetic overflow error occurs, and an exception that resembles the following is thrown:
Data was written to the Data Warehouse staging area but processing failed on one of the subsequent operations.
Exception 'SqlException': Sql execution failed. Error 8115, Level 16, State 2, Procedure RelationshipChange, Line 180, Message: Arithmetic overflow error converting expression to data type int.

Issue 8
You experience poor performance when Operations Manager searches for attributes. Additionally, when doing more than one search in a row, the cache file (momcache.mdb) keeps increasing.

Issue 9
SCX agent generates lots of warnings in the log file when a user is using Security-Enhanced Linux (SELinux).

Issue 10
By default, the Rpcimap monitor for Red Hat Enterprise Linux 6 is disabled.

GOOD NEWS!!!!! A new version of the SCOM Management Pack for Exchange 2013 is released!

EXCHANGE 2013 Management Pack

This contains new performance rules and views here are some below to give an idea of what comes with:

Collecting performance data for Microsoft Exchange Server 2013 components

This Management Pack collects following performance metrics:

• Exchange 2013 Server

  • Exchange 2013 Database: I/O Database Reads Average Latency (ms)

  • Exchange 2013 Database: I/O Database Writes Average Latency (ms)

  • Exchange 2013 Database: I/O Log Reads Average Latency (ms)

  • Exchange 2013 Database: I/O Log Writes Average Latency (ms)

  • Exchange 2013 Database: Page Fault Stalls/sec

  • Exchange 2013 Server: Average Disk sec/Read

  • Exchange 2013 Server: Average Disk sec/Write

  • Exchange 2013 Server: Disk Free Space (MB)

  • Exchange 2013 Server: Disk Reads/sec

  • Exchange 2013 Server: Disk Size (MB)

  • Exchange 2013 Server: Disk Writes/sec

  • Exchange 2013 Server: Transport Queue Database Size (MB)

  • Exchange 2013 Server: Client Connections Count

  • Exchange 2013 Server: ActiveSync Pending

  • Exchange 2013 Server: Avg. Authentication Latency (ms)

  • Exchange 2013 Server: E2E Deliver <=90sec

  • Exchange 2013 Server: E2E Latency Percentile 95

  • Exchange 2013 Server: E2E Latency Send to External <=90sec

  • Exchange 2013 Server: EWS Response Time (ms)

  • Exchange 2013 Server: LDAP Search Time (ms)

  • Exchange 2013 Server: Messages Received/sec

  • Exchange 2013 Server: Messages Sent/sec

  • Exchange 2013 Server: Outstanding Proxy Requests

  • Exchange 2013 Server: HTTP Proxy Failure Rate (%)

  • Exchange 2013 Server: HTTP Proxy Requests/sec

  • Exchange 2013 Server: Queues by Type Count

  • Exchange 2013 Server: Queue Length

  • Exchange 2013 Server: Requests/sec

  • Exchange 2013 Server: Avg RPC Latency (ms)

  • Exchange 2013 Server: SMTP Bytes Received/Sec

  • Exchange 2013 Server: SMTP Bytes Sent/Sec

  • Exchange 2013 Server: SMTP Connections (Current)

  • Exchange 2013 Server: Store Messages Submitted/sec

  • Exchange 2013 Server: System Total Memory (MB)

  • Exchange 2013 Server: CPU Utilization (%)

  • Exchange 2013 Server: Private Memory Size (MB)

  • Exchange 2013 Server: Private Memory Size (%)

• Exchange 2013 Windows Service

  • Exchange 2013 Windows Service: CPU Utilization (%)

  • Exchange 2013 Windows Service: Pool Non Paged Memory Size (MB)

  • Exchange 2013 Windows Service: Pool Paged Memory Size (MB)

  • Exchange 2013 Windows Service: Private Memory Size (MB)

  • Exchange 2013 Windows Service: Private Memory Size (%)

• Exchange 2013 IIS Application Pool

  • Exchange 2013 IIS App Pool: Active Requests

  • Exchange 2013 IIS App Pool: Private Memory Size (%)

  • Exchange 2013 IIS App Pool: CPU Utilization (%)

  • Exchange 2013 IIS App Pool: Pool Non Paged Memory Size (MB)

  • Exchange 2013 IIS App Pool: Pool Paged Memory Size (MB)

  • Exchange 2013 IIS App Pool: Private Memory Size (MB)

• Exchange 2013 Mailbox Database Copy

  • Exchange 2013 Mailbox Database: Available Space (MB)

  • Exchange 2013 Mailbox Database: Index Size (MB)

  • Exchange 2013 Mailbox Database: I/O Database Reads Average Latency (ms)

  • Exchange 2013 Mailbox Database: I/O Database Writes Average Latency (ms)

  • Exchange 2013 Mailbox Database: I/O Log Reads Average Latency (ms)

  • Exchange 2013 Mailbox Database: I/O Log Writes Average Latency (ms)

  • Exchange 2013 Mailbox Database: Transaction Log Available Space (MB)

  • Exchange 2013 Mailbox Database: Transaction Log Size (MB)

  • Exchange 2013 Mailbox Database: Database Page Fault Stalls/sec

  • Exchange 2013 Mailbox Database: Database Size (MB) 

Collecting performance data for Microsoft Exchange Server 2013 Organization

This Management Pack defines several “synthetic” performance metrics which shows overall Exchange Organization performance. These metrics are calculated as an aggregation of performance values collected at Exchange Servers.

• Exchange 2013 Organization

  • Exchange 2013 Organization Synthetic: Mailbox Count

  • Exchange 2013 Organization Synthetic: Client Connections Count

  • Exchange 2013 Organization Synthetic: E2E Deliver <=90sec

  • Exchange 2013 Organization Synthetic: E2E Latency Send to External <=90sec

  • Exchange 2013 Organization Synthetic: Mailbox Database Size (MB)

  • Exchange 2013 Organization Synthetic: Messages Received/sec

  • Exchange 2013 Organization Synthetic: Messages Sent/sec

  • Exchange 2013 Organization Synthetic: Store Messages Submitted/sec

http://support.microsoft.com/kb/2965445

Here are the fixes:

• A deadlock condition occurs when a database is connected after an outage. You may experience this issue may when one or more HealthServices services in the environment are listed as Unavailable after a database goes offline and then comes back online.
• The Desktop console crashes after exception TargetInvocationException occurs when the TilesContainer is updated. You may experience this issue after you leave the console open on a Dashboard view for a long time.
• The Password expiration monitor is fixed for logged events. To make troubleshooting easier, this fix adds more detail to Event IDs 7019 and 7020 when they occur.
• The Health service bounces because of high memory usage in the instance MonitoringHost: leak MOMModules!CMOMClusterResource::InitializeInstance. This issue may be seen as high memory usage if you examine monitoringhost.exe in Performance Monitor. Or, the Health service may restart every couple of days , depending on the load on the server.
• The Health service crashes in Windows HTTP Services (WinHTTP) if the RunAs account is not read correctly.
• Windows PowerShell stops working with System.Management.Automation.PSSnapInReader.ReadEnginePSSnapIns. You may see this issue as Event ID 22400 together with a description of "Failed to run the Powershell script."
• The PropertyValue column in the contextual details widget is unreadable in smaller widget sizes because the PropertyName column uses too much space.
• The update threshold for monitor "Health Service Handle Count Threshold" is reset to 30,000. You can see this issue in the environment, and the Health Service Handle Count Threshold monitor is listed in the critical state.
• An acknowledgement (ACK) is delayed by write collisions in MS queue when lots of data is sent from 1,000 agents.
• The execution of the Export-SCOMEffectiveMonitoringConfiguration cmdlet fails with the error "Subquery returned more than 1 value.”
• The MOMScriptAPI.ReturnItems method can be slow because a process race condition may occur when many items are returned, and the method may take two seconds between items. Scripts may run slowly in the System Center Operations Manager environment.
• When you are in the console and click Authoring, click Management Pack, click Objects, and then click Attributes to perform a Find operation, the Find operations seems unexpectedly slow. Additionally, the Momcache.mdb file grows very large.
• A delta synchronization times out on SQL operations with Event ID 29181.
• Operations Manager grooms out the alert history before an alert is closed.
• The time-zone settings are not added to a subscription when non-English display languages are set. Additionally, time stamps on alert notifications are inaccurate for the time zone.
• Web Browser widget requires the protocol (http or https) to be included in the URL.
• You cannot access MonitoringHost's TemporaryStoragePath within the PowerShell Module.
• The TopNEntitiesByPerfGet stored procedure may cause an Operations Manager dashboard performance issue. This issue may occur when a dashboard is run together with multiple widgets. Additionally, you may receive the following error message after a time-out occurs:
  
  
  [Error] :DataProviderCommandMethod.Invoke{dataprovidercommandmethod_cs370}( 000000000371AA78 )
  An unknown exception was caught during invocation and will be re-wrapped in a DataAccessException. System.TimeoutException: The operation has timed out.
  at Microsoft.EnterpriseManagement.Monitoring.DataProviders.RetryCommandExecutionStrategy.Invoke(IDataProviderCommandMethodInvoker invoker)
  at Microsoft.EnterpriseManagement.Presentation.DataAccess.DataProviderCommandMethod.Invoke(CoreDataGateway gateWay, DataCommand command)

I was working with a customer when the need to schedule their maintenance was brought up.  As we all know this is a very sore spot for SCOM that we cannot do!  We had the great idea of using Orchestrator to do the dirty work for us as long as our customers filled out a CSV document (correctly :)). 

In this blog post I will show you how to build that Maintenance Mode from CSV File runbook and also a secondary runbook that will query your database every 5 minutes to initiate the scheduled maintenance.

High-Level Steps

1) Create Databases (one to store our scheduled Maintenance requests and the other for reporting).
2) Create File location that you will store the Template.csv and where your users will save their requests.
3) Create file location for the completed requests to be moved to.
4) Create MM from CSV file runbook (this can also be created to pull from SharePoint 2013)
5) Create the Schedule Maintenance Mode runbook
6) Test and release to your customers!

=================================================

Let’s start!

1. We first need to create our 2 Databases.  You can always consolidate these into one but for my demo purposes I kept them separate.  I did use the same script to create the tables on both databases so the tables are mimics of each other.

Here is what my structure looks like:



Script to create the table(if you do use 2 databases, just change the USE [location]:
USE [ScheduledMaintMode]

GO

 /****** Object:  Table [dbo].[MMInfo]    Script Date: 4/21/2014 12:32:50 PM ******/

SET ANSI_NULLS ON
GO 
SET QUOTED_IDENTIFIER ON
GO 

CREATE TABLE [dbo].[MMInfo](
        [ServerName] [nvarchar](max) NOT NULL,
        [StartDate] [date] NOT NULL,
        [StartTime] [nvarchar](50) NOT NULL,
        [Duration] [nvarchar](50) NOT NULL,
        [RequestedBy] [nvarchar](max) NULL,
        [RepeatSchedule] [nchar](10) NULL,
        [NumberOfOccurences] [nchar](10) NULL,
        [CompletedOn] [nchar](10) NULL
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO

2. Create our File locations.

The first location will be where we are going to store our MaintMode_Template.csv and where our users will also drop their requests.  We need to make sure and educate the users to open the template and SAVE AS MaintMode_[Their UserID].csv.  Now in my lab I just created this path local to my SCORCH server but you can create this on a file share that everyone will have access to.


3.  Now we need to create another location so that our completed files have a place for auditing.


You can see from the image above that we move and rename the file Month_Day_Year_FileName.  This allows us to see when it was run.
RUNBOOK #1: CSV FILE MM

Now we have done the first 2 steps so we need to open up SCORCH and create our Maintenance Mode folder.  Under this folder we will create 2 sub folders. 
One for the CSV File Maintenance Mode runbook and the other for Scheduled Maintenance Mode.

In our CSV file MM we need to right click and create new runbook.  When completed it will look like below:

So let’s dig in :) We start with our Monitor File Activity.  This is located under the File Management folder on the right side.


Monitor File:
Once you have this go ahead and drag onto our new Runbook.  Double click it and set the parameters to mimic below:

*We need to make sure that the relation in the filter says “Does not match pattern” and value = *Template*. 
In doing this, it will allow us to always keep our Template in the location without ever getting picked up by the runbook and moved.

Next we need to drag out the Run .Net Activity onto our screen so that we can parse the .csv data.  This activity is located under the System Activities.  To save time we will drag this icon on 5 times. ( it seems crazy but trust me ;) )


Once you have dragged all of them out make them look like our parent image and connect them all up.  To connect them on the right of the Monitor File activity in the runbook there will be a small arrow.  Hover over this until your cursor becomes a cross, then click and connect the activities.  This allows us to pull published data from anything connected in front of the selected activity.

Process Initial Data, we need to make sure the type is PowerShell (in fact all of the .Net Activities we use will be PowerShell).
You will notice that $filepath has some nasty value, this is because it is published data from Monitor File activity.
 
$FilePath= '\`d.T.~Ed/{A122FD91-6B43-4C55-978D-85142E54D074}.FileName\`d.T.~Ed/'
$Data = Import-CSV $FilePath
$Array = @()
ForEach ($agent in $Data)

{
 $ServerName = $agent.'ServerName(FQDN)'
 $Starttime = $agent.'StartTime(military & 5 min increments)'
 $StartDate = $agent.'StartDate(MM/DD/YYYY)'
 $Duration = $agent.'Duration(mins)'
$Repeat = $agent.'Repeat(Yes or No)'
$Type = $agent.'Type'
$Occurences = $agent.'Number Of Occurences'
$Owner = $agent.'Requested By'

     if($Starttime.ToLower() -ne 'now')

    {
        $RunNow = 'False'
    }
    Else
    {
        $RunNow = 'True'
    }

    $str = $ServerName +","+$StartDate+","+$Starttime+","+$Duration+","+$Repeat+","+$Type+","+$Occurences+","+$Owner+","+$RunNow
    $Array += $str+";"

}

 The image should look like this:


To reference published data from another activity, 1^st they must be connected and 2^nd you need to right click à subscribe à Published Data. This will by default pull the first connected activity, but just hit the drop down and you can pull from other activities as long as they are before the activity you’re on.  Also if you don’t see some common data, check the ‘Show common Published Data’(this is the data we will use to rename your file):



Once that is completed we need to publish some data from this activity that will be used down the road.



Now before we move to the next .Net Activity which is Agent Validation, we need to move our file and rename it:


These are located under the same folder as Monitor File.  We just need to drag them onto our runbook and connect them similar to above.  As for the parameters
Move the MM File:

Rename the File ( all I have done is right click the Destination properties and hit expand allowing for much easier editing/modifing):


Now let’s get back to Agent Validation Activity.  This is where we will validate the data the user entered is correct and also has a SCOM agent on it.  Here you use the Exchange integration pack to notify the User and SCOM team that there are boxes that are either incorrectly entered or don’t have a SCOM agent.
Agent Validation:


Script( you will need to modify the SCOM connection to your Mgmt. Server!!!):
#Import OM Module
Import-Module OperationsManager;
New-SCOMManagementGroupConnection [mgmt. server here]

$ValidArray = @()
$BadServerArray = @()

$Servers = '\`d.T.~Ed/{BB1A40F6-93D4-45FE-AB52-2A9DF8065A24}.Array\`d.T.~Ed/'
$Agents = $Servers.Split(';')
foreach ($line in $Agents)
{
If (![string]::IsNullOrEmpty($line))
    {
$line = $line.Trim()       
$processsplit = $line.split(',')
        $servername = $processsplit[0].Trim()
         $validAgent = (get-scomagent| where {$_.Displayname -eq $servername}).Displayname
        If (![string]::IsNullOrEmpty($servername))
           {
              if ($validAgent -match $servername)
                 {
                   $ValidArray += $line
                 }
         else
          {
                $BadServerArray += $line
          }
       }
    }
}

Publish Data:


Process Valid Agents, this where we will break apart our Good Array and use this data to either kick off immediately or schedule.
Details:


Downside is we now need to publish all of this :(

Published Data:


Nice!  Now the majority of the hard work is done!  Now we need to connect the Maintenance Mode activity from the SCOM IP. 
Locate this and drag onto the runbook and connect to Process Valid Agents. For this connector we need to add some criteria!

To do this we need to double click on the connecting line between the 2 activities and add our own criteria:


Start MM activity

*You need to make sure your Monitor field is like above. If not then Run Now MM will never work :(*  
Format:
{ServerName from “Process Valid Agents”} : Microsoft.Windows.Computer:{ServerName from “Process Valid Agents”}  -->NOTICE WHERE THERE ARE SPACES AND NOT!!!!

In my example I have this going to the Event Log, this is just another form of auditing you can apply.  Have SCOM alert so that you can report on it later when it finds MM events.

OK, now on to the one off scheduling.  This is just for I need Server A to go into Maintenance Mode next week and that is all.  We first need to get the Write to DB activity from Utilities.


Drag this icon onto the runbook twice and connect like below:


On the connector from Process Valid Agents to Schedule Maintenance Mode it needs to have this criteria in the Include:


Exclude:


We are doing to make sure that we are excluding our agents that need to have schedules setup. 

Schedule Maintenance Mode Activity:
Details:


Connection tab:

*You will need to modify the Server field to match where you created your DB(s) in the earlier steps.* 
**Also make sure that you gave your SCORCH service rights to the DB, because in the next runbook he will be creating, deleting, and moving records!**

Now let’s move to the recurring schedule portion of our runbook.  Here we are offering the users the option of daily, weekly, or monthly and in that last .Net Activity you will see the PowerShell we are using to accomplish this. First we need to start with the connector:
Include:

Exclude:


Create Schedule Activity:
Details:


Script:
$StartDate = '\`d.T.~Ed/{73B75251-B48E-45E8-B5D8-1B6DEC647D63}.StartDate\`d.T.~Ed/'
$strdate = get-date $StartDate
$Type = '\`d.T.~Ed/{73B75251-B48E-45E8-B5D8-1B6DEC647D63}.Type\`d.T.~Ed/'
$Occurences = '\`d.T.~Ed/{73B75251-B48E-45E8-B5D8-1B6DEC647D63}.Occurences\`d.T.~Ed/'
$Servername = '\`d.T.~Ed/{73B75251-B48E-45E8-B5D8-1B6DEC647D63}.Servername\`d.T.~Ed/'
$StartTime = '\`d.T.~Ed/{73B75251-B48E-45E8-B5D8-1B6DEC647D63}.StartTime\`d.T.~Ed/'
$Owner = '\`d.T.~Ed/{73B75251-B48E-45E8-B5D8-1B6DEC647D63}.Owner\`d.T.~Ed/'
$Duration = '\`d.T.~Ed/{73B75251-B48E-45E8-B5D8-1B6DEC647D63}.Duration\`d.T.~Ed/'

$array = @()

If($Type -eq 'Daily')
{
    $NewDate = $strdate
    if ($Occurences -gt '1')
    {
        $str = $Servername +","+($NewDate).ToShortDateString()+","+$Starttime+","+$Duration+","+$Type+","+$Occurences+","+$Owner
        $Array += $str +";"
        2..$Occurences | foreach-object{`
        $NewDate =$NewDate.AddDays(1)
        $Arraydate = $NewDate.ToShortDateString()
        $str = $Servername +","+$Arraydate+","+$Starttime+","+$Duration+","+$Type+","+$Occurences+","+$Owner
        $Array += $str +";"
        }
    }
}

If($Type -eq 'Weekly')
{
    $NewDate = $strdate
    if ($Occurences -gt '1')
    {
        $str = $Servername +","+($NewDate).ToShortDateString()+","+$Starttime+","+$Duration+","+$Type+","+$Occurences+","+$Owner
        $Array += $str +";"
        2..$Occurences | foreach-object{`
        $NewDate =$NewDate.AddDays(7)
        $Arraydate = $NewDate.ToShortDateString()
        $str = $Servername +","+$Arraydate+","+$Starttime+","+$Duration+","+$Type+","+$Occurences+","+$Owner
        $Array += $str +";"
        }
    }
} 

If ($Type -eq 'Monthly')
{
$NewDate = $strdate
$DOW = $strdate.DayOfWeek
    if ($Occurences -gt '1')
    {
        $str = $Servername +","+($NewDate).ToShortDateString()+","+$Starttime+","+$Duration+","+$Type+","+$Occurences+","+$Owner
        $Array += $str +";"
        2..$Occurences | foreach-object{`
        $NewDate =$NewDate.AddMonths(1)
        if($NewDate.DayOfWeek -ne $strdate.DayOfWeek)
        {
            $n = 0
            do{
            $NDate = $NewDate.AddDays($n)
            $n++}
            Until ($NDate.DayOfWeek -eq $DOW)           

            if(($nDate.Day - $strdate.Day) -gt '5')
            {
                $n = 0
                do{
                $NDate = $NewDate.AddDays(-$n)
                $n++}
                Until ($NDate.DayOfWeek -eq $DOW)
            }
       }            

        $Arraydate = $NDate.ToShortDateString()      

        $str = $Servername +","+$Arraydate+","+$Starttime+","+$Duration+","+$Type+","+$Occurences+","+$Owner
        $Array += $str +";"
        }
    }
}

Published Data:

Process Scheduled Jobs Activity:
Details:


Script:
$process = '\`d.T.~Ed/{1B802AEB-4D49-4DFB-8C43-7FC61CE93400}.array\`d.T.~Ed/'
$processsplit = $process.split(',')
$servername = $processsplit[0]
$StartDate = $processsplit[1]
$StartTime = $processsplit[2]
$Duration = $processsplit[3]
$Type = $processsplit[4]
$Occurences = $processsplit[5]
$Owner = $processsplit[6]
if ($Owner -match ";")
{
$Owner = $Owner.Replace(";","")
}

Published Data:


Now the very last activity of the CSV File MM runbook, :)!!!

Schedule Future MM Activity:

Details:


Connection tab:

*Make sure to modify for your SQL Server from the first step.*


RUNBOOK #2: SCHEDULED MAINT. MODE
Overview of runbook:


For this runbook we will run on a 5 minute interval as we don’t want to overwhelm our SCORCH instance and also the SCOM instance.  We need to start with the Monitor Date/Time Activity. This activity will be located under Scheduling.

5 Min. Interval Activity:


We now need to get the Query Database activity under the same location as the Write to Database activity.  Drag this activity to your runbook and connect the 2 activities.
We will also need to drag 3 more of them for activities like you see in the main screen shot.

Query Database Activity:
Details:


Query:
select ServerName, Duration, StartTime,StartDate,RequestedBy from MMInfo
where StartDate like (SELECT FORMAT(CURRENT_TIMESTAMP, 'yyyy-MM-dd'))
and StartTime Between (SELECT FORMAT(CURRENT_TIMESTAMP, 'HH:mm')) and DATEADD(MINUTE, 5, (SELECT FORMAT(CURRENT_TIMESTAMP, 'HH:mm')))

Connection tab:

*Be sure to adjust for your SQL Server

Next we need to drag a Run .Net Activity to our runbook and connect it from the Query Database activity. We are also going to be adding some criteria to the connector to stop the run book if no rows are returned from our query.


Process Query Results Activity:
Details:


Script:
$process = '\`d.T.~Ed/{6BEF2F48-9A68-41D1-A1C1-E3A16B6C10A5}.Full-line\`d.T.~Ed/'
$processsplit = $process.split(';')
$servername = $processsplit[0]
$Duration = $processsplit[1]
$StartTime = $processsplit[2]
$StartDate = $processsplit[3]
$RequestedBy = $processsplit[4]

 
Published Data:


Now we have broken apart all the data from our query and we need to not only move it to our completed from scheduled db but also kick off Maintenance Mode!!!  Kind of the whole reason you’re hereJ!!!

Start Maintenance Mode Activity:
Details:

*Once again just like in the other runbook you need to be very cautious of the Monitor field.  If this isn’t right you will never start MM on your agents.
**You will also see two other activities coming off of Start Maintenance Mode, these are just extra steps for notifying that MM has started and auditing.

Update MM Tables Activity:
This activity will be used to move the selected rows from Scheduled to Completed DBs.
Details:

*If you used only one DB for your tables make sure you account for that in our activities!

Script:
Insert into [CompletedMaintMode].[dbo].[MMInfo]
Select *from MMInfo
where StartDate = '\`d.T.~Ed/{EF99E84D-DEEE-442F-8B11-C468131532E9}.StartDate\`d.T.~Ed/'
and StartTime = '\`d.T.~Ed/{EF99E84D-DEEE-442F-8B11-C468131532E9}.StartTime\`d.T.~Ed/'
and ServerName = '\`d.T.~Ed/{EF99E84D-DEEE-442F-8B11-C468131532E9}.servername\`d.T.~Ed/'
and Duration = '\`d.T.~Ed/{EF99E84D-DEEE-442F-8B11-C468131532E9}.Duration\`d.T.~Ed/'
and RequestedBy = '\`d.T.~Ed/{EF99E84D-DEEE-442F-8B11-C468131532E9}.RequestedBy\`d.T.~Ed/'

Connection Tab:


We need to move onto the Update Completed.  This activity will update our rows notifying us for our reports when the MM was actually executed.

Update Completed Activity:
Details:


Script:
Update [CompletedMaintMode].[dbo].[MMInfo]
set [CompletedMaintMode].[dbo].[MMInfo].CompletedOn = (SELECT FORMAT(CURRENT_TIMESTAMP, 'yyyy-MM-dd'))
where StartDate = '\`d.T.~Ed/{EF99E84D-DEEE-442F-8B11-C468131532E9}.StartDate\`d.T.~Ed/'
and StartTime = '\`d.T.~Ed/{EF99E84D-DEEE-442F-8B11-C468131532E9}.StartTime\`d.T.~Ed/'
and ServerName = '\`d.T.~Ed/{EF99E84D-DEEE-442F-8B11-C468131532E9}.servername\`d.T.~Ed/'
and Duration = '\`d.T.~Ed/{EF99E84D-DEEE-442F-8B11-C468131532E9}.Duration\`d.T.~Ed/'
and RequestedBy = '\`d.T.~Ed/{EF99E84D-DEEE-442F-8B11-C468131532E9}.RequestedBy\`d.T.~Ed/'

Connection:


Now that we have updated our completed we can now delete all completed records from our Scheduled DB.  This will keep it clean and hopefully minimize any confusion for users, reporting, etc.

Delete from ScheduledMM Activity:
Details:


Script:
Delete from MMInfo
where StartDate = '\`d.T.~Ed/{EF99E84D-DEEE-442F-8B11-C468131532E9}.StartDate\`d.T.~Ed/'
and StartTime = '\`d.T.~Ed/{EF99E84D-DEEE-442F-8B11-C468131532E9}.StartTime\`d.T.~Ed/'
and ServerName = '\`d.T.~Ed/{EF99E84D-DEEE-442F-8B11-C468131532E9}.servername\`d.T.~Ed/'
and Duration = '\`d.T.~Ed/{EF99E84D-DEEE-442F-8B11-C468131532E9}.Duration\`d.T.~Ed/'
and RequestedBy = '\`d.T.~Ed/{EF99E84D-DEEE-442F-8B11-C468131532E9}.RequestedBy\`d.T.~Ed/'

Connection:



7. Validation time!!!

Create my file:

Drop onto my scheduled folder and watch the completed to see it working :)



Now I look back at the runbook and I see happiness:


Now let’s check the database to see our schedule!

I will modify one of the rows to be today and start at 3:58PM

Verify that that my server isn’t in Maintenance Mode before job kicks off:

Back to the Scheduled Maint. Mode runbook and watch for the 3:55 job run and pick it up.

 
So we see it was picked up from the 3:55 job so let’s check the DB again.

Only 3 rows!!

Now we can check the Completed table:
 

Now back to SCOM to validate!!!
As you can see the time difference is 40 minutes and the comment matches exactly what we have for our Maintenance Mode activities:

Now this is just the Ford Pinto of how to schedule maintenance mode, you can always use Tim McFadden's Cadillac version which is VERY nice!!!

I am always open to hearing about how you do it at your company or if you think there are ways to improve this! 
Please leave comments on either, I would love to read them.

Disclaimer:
This example is provided “AS IS” with no warranty expressed or implied. Run at your own risk.
The opinions and views expressed in this blog are those of the author and do not necessarily state or reflect those of Microsoft.

**Always test in your lab first**  Do this at your own risk!! The author will not be held responsible for any damage you incur when making these changes!

MaintenanceModeTemplate.csv