Common question I keep getting asked by my customers is: How do I set Orchestrator Operator privileges in specific Runbook contexts?

This post is all about providing you with a simple and precise set of guidelines to enable to answer the question above.

The Scenario:

"WSUS Operators" Domain Group needs to only gain Operator access to Start/Stop/Assess Runbook execution on the WSUS Runbook structure within Orchestrator while using the Orchestrator Console.

Enabling listing of Runbooks\Folders

This is required otherwise nothing is listed on the Orchestrator Console.

Set User or Group (recommended) on "Runbooks" root folder on the Designer Tool

• Right-click Runbooks > Permissions

• Click Add to include the Group ("WSUS Operators" in this example)

• Click Advanced
• Select the group you added
• Click Edit
• Click Show Advanced Permissions
• Set permissions as follows:

• Click Ok > Apply > OK > OK
• Permissions should then show as follows:

NOTE: To make changes quickly effective run the following SQL statement against your Orchestrator Database:

TRUNCATE TABLE [Microsoft.SystemCenter.Orchestrator.Internal].AuthorizationCache

Setting specific Runbook Execution privileges

Set permissions for User or Group (recommended) on Specific Runbooks\Folders (Read, List, Publish)

• Right-click the specific folder (in this case "WSUS") > Permissions

• Click Add to include the Group ("WSUS Operators" in this example)

• Click Advanced
• Select the group you added
• Click Edit
• Click Show Advanced Permissions
• Set permissions as follows:

• Click Ok > Apply > OK > OK
• Permissions should then show as follows:

NOTE: To make changes quickly effective run the following SQL statement against your Orchestrator Database:

TRUNCATE TABLE [Microsoft.SystemCenter.Orchestrator.Internal].AuthorizationCache

• The Orchestrator Console should show as follows, hence preventing access to the other Runbook contexts for a given Operator member of the "WSUS Operators" group:

The above is shown regardless of the additionally existent Runbooks\Folders:

Hope the above helps!

 #Get the current version of the VMM Server

 $VMMServer = "VMMSERVERNAME"

 Get-SCVMMServer -ComputerName $VMMServer | ft Name, ProductVersion

Get-SCVMMManagedComputer  | ft Name, Role, AgentVersion

Get-SCVMMManagedComputer | ? {$_.AgentVersion -ne "4.0.1381.0"} | ft Name, Role, AgentVersion

$VMMServer = "INSERTVMMSERVERNAMEHERE" $HostManAccount = "INSERTHOSTMANAGEMENTACCOUNTNAMEHERE"

$VMMBuild = Get-SCVMMServer -ComputerName $vmmserver $VMMBuildVersion = $VMMBuild.ProductVersion

#Set the variables for the Run As Account $credential = Get-SCRunAsAccount -Name $HostManAccount

#Create the list of target Servers by identifying those that are out of date and put them into an array $managedComputers = Get-SCVMMManagedComputer | ? {$_.AgentVersion -ne $VMMBuildVersion}

#Deploy the Agent update to all servers identified as being out of date Foreach($managedComputer in $managedComputers) { Update-SCVMMManagedComputer -Credential $credential -RunAsynchronously -VMMManagedComputer $managedComputer }

https://support.microsoft.com/en-us/help/10082/troubleshooting-pxe-boot-issues-in-configuration-manager-2012

Rob York

Premier Field Engineer

Firstly, some background.  Way back when, ConfigMgr 2012 SP1 introduced a feature called "Client Notification".  This added an option on the right click context menu for a client or a collection that granted the administrator the ability to trigger a machine or user policy retrieval and evaluation cycle.  Admins may be forgiven for thinking that this is a “push” action to the client, as it’s actually a client initiated “pull” after the admin triggers it.  This is ConfigMgr after all – the client requests everything.  Quite simply, the client is notified to start an action normally controlled by a schedule.

This feature was designed in order for vital deployments to be sent to clients a lot quicker and to save them waiting for their computer policy retrieval cycle, set to every 60 minutes by default.  Scenarios this would help with would be a zero-day patch, an urgent anti-malware definition or some other kind of scripted fix that needed to go out to all clients very quickly.

I won’t go into details about the ins and outs of Client Notification, although you can read the Product Group’s original blog about the feature release here.  In a nutshell, clients communicate over TCP or HTTP every 15 minutes to confirm they’re still online.  Once an action is triggered using Client Notification, only clients that have reported back as “online” will be notified to begin an action.  This means that your infrastructure won’t be wasting resources trying to open up individual WMI connections to clients in order to trigger these actions.  Other tools that seem to perform these actions can be quite costly in terms of wasted resources, especially when targeting a large collection.    That’s why we would suggest using the out of the box tools whenever possible.

“But Ricky, sometimes I need to do more than just a policy retrieval on a group of computers and using other tools to perform this seems like the only way!” 

That’s a very good point, and the very reason that the list of actions that can be performed in the Client Notification has now been extended in the 1602 release.  Check it out…

You can see along with computer and user policy retrieval; we now have the ability to collect discovery data, start a hardware or software inventory cycle, evaluate any deployed applications, or start a software update deployment evaluation too.  As stated above, triggering any of these actions will see that only clients that have reported back in the last 15 minutes will be given the notification to begin the chosen action. 

One more feature that was introduced that takes advantage of the “keep-alive” message sent by clients every 15 minutes is being able to see if a client is online or not when viewing it in a collection.  Have a look…

The grey icon represents that the client is offline, and the green icon shows that the client’s keep-alive message has been received in the last 15 minutes.

I hope this helps explain this small feature introduction, which seems to have slipped under most admins’ radars.  Remember, if you have any suggestions about ConfigMgr, please log them at the UserVoice site and get those votes assigned to any other features you like the look of. 

As always, please leave any comments or questions below.

Ricky

So the challenge for me was to deploy a Windows Virtual Machine into Azure and as part of that deployment, enable the Microsoft Monitoring Agent to connect into the Operations Management Suite. And I wanted to use the new Resource Manager functionality rather than the classic service management model - more on the differences here.

You’ll need to have Visual Studio 2013 or 2015 and along with Azure PowerShell, Microsoft Azure Authoring Tools and Microsoft Azure Tools for Visual Studio (version 2.8 at the time of writing).

The create a new project and choose Azure Resource Group

I’ll keep it simple and choose Windows Virtual Machine

If you have never used this before, I’ll run through a quick example.

If you take a look at the left hand side of the Visual Studio, then you’ll see four parameters (which we’ll fill in when we deploy the virtual machine and then a list of variables where we can set the vmName, size etc.

Then we need to right click on the project and create a new deployment.

I’ve created a new Resource Group as below:

Then I need to click Edit Parameters to provide the parameter information

Once that is done, click deploy … and we’ll soon have ourselves a windows virtual machine as part of a resource group

But, we don’t have the Microsoft Monitoring Agent installed. We can’t (as of November 2015) enable the Microsoft Monitoring Agent in the Azure Preview Console for Resource Manager Virtual Machines which leaves us the options of retrospectively scripting a solution or, even better, enabling the Microsoft Monitoring Agent as part of the json template.

I’ve taken my inspiration from here but I couldn’t get this to work due to issues with the formatting and the location parameter

https://www.danielstechblog.de/azure-azure-stack-iaas-vm-arm-mma-operational-insights-template-integration/

In the parameters section we need to add the following (which will allow us to state the name of the extension we are installing and the workspace id and key):

"vmExtensionMonitoringAgent": {

"type": "string",

"metadata": {

"description": "Monitoring Agent extension name"

}

 },

"monitoringAgentWorkspaceID": {

"type": "string",

"metadata": {

"description": "Monitoring Agent Workspace ID extension name"

}

 },

"monitoringAgentWorkspaceKey": {

"type": "securestring",

"metadata": {

"description": "Monitoring Agent Workspace Key extension name"

}

 }

Then we need to add the following resources

{

"apiVersion": "2015-06-15",

"type": "Microsoft.Compute/virtualMachines/extensions",

"name": "[concat(variables('vmName'),'/', parameters('vmExtensionMonitoringAgent'))]",

"location": "[resourceGroup().location]",

"dependsOn": [

"[concat('Microsoft.Compute/virtualMachines/', variables('vmName'))]"

],

"properties": {

"publisher": "Microsoft.EnterpriseCloud.Monitoring",

"type": "MicrosoftMonitoringAgent",

"typeHandlerVersion": "1.0",

"settings": {

"workspaceId": "[parameters('monitoringAgentWorkspaceID')]"

},

"protectedSettings": {

"workspaceKey": "[parameters('monitoringAgentWorkspaceKey')]"

}

 }

 }

Or copy and paste the following to replace the existing code.

I have then updated the vmname and some other naming details.

I can now deploy the template, creating a new resource group – RG123WithMMA

And now when I click on Edit Parameters, I can fill in my OMS details:

And then save and deploy.

And when the virtual machine is deployed, you have the Microsoft Monitoring Agent installed and the virtual machine appearing in the OMS workspace.

So, to review DRS replication there is nothing really that we can add to Umair Kahn’s excellent discussion on DRS here https://blogs.technet.com/b/umairkhan/archive/2014/02/18/configmgr-2012-data-replication-service-drs-unleashed.aspx.  Umair explains in his post that there are two main types of data:

• Global data, which as its name suggests is replicated between all primary sites and CAS within the hierarchy.  There is a derivation of this called Global Proxy data which is replicated from the primary site to any secondary sites under it.

• Site data, which is unique to this primary site and then is replicated up to the CAS

We then read on to find that the data that we replicate is arranged into articles and these articles are then arranged into replication groups.  The replication groups then replicate between the sites.

The transport mechanism for this data to be replicated is via a SQL Server feature called change tracking.  Quoting from Umair’s blog post:

Change tracking is to identify and keep track of what changed in the Database like a row insertion, updation or deletion. We need this as we have to now only send the changes to the CAS and not the whole global data again as both the CAS and primary are now in page.

Question can come how Change Tracking works in background –

The answer is simple for every table that is enabled for change tracking we have one more internal table change_tracking_<object_id >. When a row gets inserted in the actual table then a corresponding entry (containing) only the primary key information gets added in the internal table change_tracking_<object_id >

So, whenever a row is changed within the local database then a second table is updated with that information.  We then replicate that data to the other site.

Umair’s blog post then goes on to say:

For every successful transaction that is committed for a table we see a row in the sys.syscommittab table. The sys.syscommittab is a system table which can be referenced by the sys.dm_tran_commit_table view

So the sys.syscommittab table is what I want to spend a little time discussing.  From the database we need to look at the sys.dm_tran_commit_table view which gives us something which looks like this:

So, the change tracking process looks into this table and picks up transactions which have not been committed, locates the un-replicated transactions and then when they are replicated, updates the table.

So, then what?  Well, we keep the records in this table for 5 days (by default).  If you have looked into DRS replication then you will have also seen that DRS can recover from an outage which is shorter than 5 days – more than that, then we will re-initialise the database.  This is why we have the 5 days limit.

When a transaction is committed and it is more than 5 days old then it will be removed from sys.syscommittab.  This occurs within a SQL process that runs every 62 seconds.  This process however has a rowcount limit of 10,000 records so we are good to prune up to 14.4 million rows per day.

If you have a few days where the number of replicated transactions is greater than 14.4 million then this will not be a problem – sys.syscommittab can accommodate for this growth and the excess records will be pruned when the growth in this table is not so great (for example, over a weekend).  However, what happens if there never really is a quiet time in your site?

Well, sys.syscommttab will do what it is supposed to do and contain the older records until it is able to prune them.  Left unchecked then the growth in this table can adversely affect performance of replication, but this will likely be a slow-burn for several months before you see it.  If you are in that situation then when you run the SQL below to see the busiest SQL statements then towards the top you will see not the usual stored procedures that you expect to see but SQL code including sys.syscommittab showing.

At this stage, while it’s nothing to panic about it would be best to log a support incident with Microsoft in order to run the correct SQL statements to bring things back into order.

Much better however is to simply monitor the growth of entries in sys.syscommittab and ensure that you are maintaining a healthy equilibrium.  With my customer we simply added a command select count (*) from sys.dm_tran_commit_table and trend this in SCOM, with an alert if the value goes above 100 million rows (we based this on 14.4 million is within bounds for their 5 days retention plus some headroom for those occasional busy times).  My colleague Rob York however took this and wrote a much more professional piece of code below:

Lastly, if you do see that these are an issue for you then logging a call with Microsoft Support will be the best way to bring your environment back into line.

The connector itself seemed to be targeted at the all management server resource pool (AMSRP) but no alerts got updated to be forwarded when the RMSe was down.

On investigation we noted that the connector subscription created which resides in the Notifications Internal Library management pack was targeted at the Root Management Server. We tested changing it over to target the AMSRP and hey presto we made the connector highly available.

Below are the steps to change the subscription over to target the AMSRP.

Please note this involves direct MP editing so caution should be taken when running in production environments.

1) Export the Notifications Internal Library and open with your favourite XML editor, in my case XML notepad.

2) Search for the connectors subscription name in the displaystrings section and find its element ID. It should start with productconnectorsubscription to make it easy for you.

3) Search in the monitoring > rules section of the MP using the elementid value and confirm the target is set as: Target="SystemCenter!Microsoft.SystemCenter.RootManagementServer"

4) Change the target to: SystemCenter!Microsoft.SystemCenter.AllManagementServersPool:

5) Save the changes and reimport the MP back in your environment. The subscription will now be running as a workflow within the AMSRP.

We are hoping this get updated at the product level so connector subscriptions automatically get targeted at the AMSRP on creation but until then the workaround can be used.

Hope this helps

Alex

I have delved around for documentation and found that public information for this is not commonplace.

This post from Rob Kuehfus has good information the changes in topology between SCOM 2007 and SCOM 2012:

https://blogs.technet.com/b/momteam/archive/2011/08/22/topology-changes-in-system-center-2012-operations-manager-overview.aspx

It covers off Resource Pool nicely but it still does not actually explain, in a way I like, how the Management Group Failover Service works.

So here is my explanation of how this works. I am aware that potentially not all permutations and combinations will be covered in writing as I did not wish to drop down a deep Rabbit Hole with every eventuality being covered in minutiae. My endeavour is that from this post you, dear reader, will be equipped with the understanding to be able to look at your own environment understanding how failover in the All Management Servers Resource Pool works and run all the permutations yourself to see if you are in “a good place”.

OK, lets get started.

SCOM can be crafted with all of its components installed onto one server. SQL instance, management server, console, SSRS – you get the picture. It will likely have lots of issues but it will work. One thing it will not be however is Fault Tolerant!

En Passant, I may be being a pedant but there is a distinction between Fault Tolerance (FT) and High Availability (HA). Here I am talking about a level of FT not making your whole SCOM solution HA.

In Rob Kuehfus Blog, towards the end it states

“Moving forward the Product Group recommendation will always be to have two management servers in a Management Group at all times.  By doing this you will always have High Availability for your management group and a much easier recovery during a disaster.”

Right, so I need to make my single server management group more fault tolerant so I will follow the advice and add a second management server. This will automatically be included in the All Servers Resource Pool (assuming I have not changed any setting on membership from the defaults). Now I will have my original server with ALL the SCOM roles and a additional second one with only the management server role installed. OK, so I am there now?

My answer is: No, but it is a lot closer!

I have gone from Bad to Good. But now to go from Good to Better….

To get from Good to Better I really need to separate out more of the SCOM roles. SQL is not (in a production level environment at any rate) recommended to be installed onto a management server. So I should split this off. I now have a management group with two management servers and a separate SQL server. This is Better.

Now, how does splitting these roles out help with the Perfect Failover Service?

To help explain this I am going to enter into one of my Analogies.

In the words of the Paul Daniels,  You'll like this, ……..

The Resource Scenario:

Every hour I like to have coffee and biscuits which I share with my office friends. So I set my schedule and on the hour get coffee and biscuits for them. This is great but the treat stops if I am ill or something – no one gets coffee and biscuits.

(The single management server has gone offline so the management group can’t function).

So to mitigate this I am going to get a friend to help (lets call him Ruben). Now Ruben could just be a passive standby. Does not do any coffee and biscuity stuff when I am around, only if I am unavailable.

(This is principle behind the clustering of a 2007 RMS)

Ideally what I really want is to spread the load rather than just doing the task myself. Having Ruben sitting around “just in case” is a waste. Lets split the task, I will always do the coffee, Ruben will always do the biscuits.

(The idea behind SCOM 2012, with the Resource Pools. The servers in the resource pools share out the management group tasks)

Back to the biscuit task! The task will run smoothly enough but what happens when either I or Rubens is unavailable? We need to have some way to confirm with each other that we will (or will not) be able to do our half of the process.

Lets set up a system that on each hour Ruben will contact me to make sure I am on for the coffee and I will confirm with a reply. Likewise I will contact Ruben to confirm he is on for the biscuits and he will reply. (OK, that bit is a little contrived but run with it).  If I don’t get a reply from Ruben I will get the biscuits as well as the coffee. And vice versa.

That has solved the problem yes? Well no, not really. What if we are both still coming for coffee and biscuits and it is just our communication connection that is down between us. We will each turn up with both coffee and biscuits (In some peoples worlds I am aware that this would not actually be a problem but for the sake of this, it is). The job has been done twice which could cause an issue.

What we need is another reference point, some mechanism by which we can confirm the situation. So we ask Kate to help. She will act as a third point of reference. If I cannot communicate to Ruben I will go to Kate for a confirmation and she will try to reach him too. If she can communicate to him she can let me know and I will continue to bring coffee knowing that Ruben will bring the biscuits. If Kate can’t get to Ruben then I will know I will need to do both tasks to bring coffee and biscuits.

Kate is our Observer, if you will.

This does need a little extra in the procedure to make it more like Mary Poppins.

If Ruben is actually in the office but cannot communicate with me he will also try Kate. If he cannot get a reply from either of us, even though he is actually available we have agreed he will stop his coffee and biscuity questing as he knows I will be doing it. And vice versa.

(Yes – this process does give us the situation that if Kate is unavailable and Ruben and I cannot “talk” we will both stop our half of the overall task and we will all be sad because no one gets coffee and biscuits . But that is a potential outcome of this process and that is how it is supposed to works). Maybe we need to make Kate’s role Fault Tolerant too – but another Rabbit hole beckons at this point so I will stop here.

There is a “Resource Pool” in place with Ruben and I in it and we have a third “Observer”, Kate, to insure Quorum.

We now have a fail over system of sorts in place.

The Perfect Failover Service in Operations Manager

If you have got this far we will now apply that to SCOM.

The tasks that a management group once performs only on the RMS in Operations Manager 2007 are now spread over the servers in the All Server Resource Pool in Operations Manager 2012.

All the servers in this pool communicate back and forth with each other to make sure that they are all available to carry out their assigned tasks. This process means that the management servers need to be on low latency connection with each other.

If we only have two management servers we need an observer node so we can get a quorum. In a two management server scenario this is the Operations Manager database.

If a management server loses communication with its partner then it will contact the database to verify that the other management server is actually offline for this observer node as well. If it finds that to be the case it will assume that its partner management server is down and take over the tasks assigned to it.

If the failed management server is actually dead then there is no real problem. If however it is still running but, say the network connection has be broken, this management server, if it cannot connect to its partner or the observer will assume that the other management server has taken all the tasks and so shut itself down (commit suicide).

The database server can exist on one of the two management servers, this gives fault tolerance for the SCOM services. We have gone from Bad to Good here.

This is however not a recommended configuration and having multiple roles on one management server will increase the risk of any one application failure taking down this management server and hence SQL as well. (No SQL and the management group will just plain not work). Move SQL off and we go from Good to Better.

Best would be things like clustered SQL, redundant network paths and all that other good stuff to get to a truly High Available solution.

Summary

In summary, the recommended configuration to get a level of Fault Tolerance in a SCOM 2012 management group you need:

Two management servers in the All servers Resource Pool and a separate Database Server as the observer node.

Three members of the system to enable quorum – a bit like in Minority Report.

Disclaimer

This posting is provided "AS IS" with no warranties, and confers no rights. Use of included utilities are subject to the terms specified at https://www.microsoft.com/info/copyright.htm.

I’ll start with what is usually a straight forward requirement – monitoring a windows log e.g. the application log – and generating an alert if a specific event id is detected. I've made it a little more complex by alerting on either of 2 event id's but it is straight forward to simplify the expression filter if required.

And I’ll assume that you have followed by initial MP authoring series (steps 1 to 3) and have created a management pack and configured the basic properties as well as having created a couple of empty folders in the solution explorer view (see below).

We’ll go through the process of right clicking Rules, Add, New Item

And then choose Empty Management Pack Fragment which for ease of identification I’ve called MultipleEvents.

Copy and paste this code into the empty fragment – you can change the following:

1. Find \ replace gd.myapp with the namespace details of your management pack

2. If necessary, update the target

    

3. Find \ Replace the event ids with the event you are looking to alert against

4. Find \ Replace EventCreate with the event source of the event you want to alert against

5. Change (if necessary) the Priority and Severity:
   
   

Severity = 2 for Critical, 1 for Warning and 0 for Informational.

Priority = 2 for High, 1 for Medium and 0 for Low.

And you should be ready to go.

Disclaimer:
All information on this blog is provided on an as-is basis with no warranties and for informational purposes only. Use at your own risk.

The opinions and views expressed in this blog are those of the author and do not necessarily state or reflect those of my employer.