System Center Operations Manager 2012: Expand Monitoring with Ease
The new System Center Operations Manager promises to bring better integrated and extended network monitoring capabilities.
Paul Schnackenburg
Monitoring any IT network environment is critical. Microsoft System Center Operations Manager (SCOM) has always provided comprehensive insight into the state of your Microsoft network. The SCOM 2007 R2 edition added Unix and Linux monitoring for keeping an eye on mixed environments.
SCOM 2012 is expected toward the end of this year. This new version will add interesting features for High Availability (HA), application performance monitoring, dashboards, network device monitoring and Java Application server monitoring.
The new version of SCOM will also have a greatly improved installation wizard. The most notable change will be that operational and data warehouse databases are created during installation. In earlier versions, you had to spin these up beforehand.
The prerequisite checker is also built into the installation wizard, which simplifies the installation process. This will highlight any errors and automatically copy the error messages. Any clipboard credentials you define are tested from within the wizard to ensure they work (see Figure 1).
Figure 1 There are several improvements made to the System Center Operations Manager 2012 installation process.
During the installation process, you have to specify the management server action account, the configuration service and data access service. You can use the same account for both roles, although from a security standpoint, that’s not a recommended best practice.
All SCOM 2012 server functions are supported to run as virtual machines (VMs). It’s recommended to run the SQL Server database on a physical server or virtual server with direct-attached disks for better performance. As with many other workloads, VM snapshots are not supported for use in conjunction with SCOM 2012.
Management and gateway servers have to run Windows Server 2008 R2 SP1, with a 2.8 GHz CPU and at least 2GB of memory. The SQL Server must be x64 2008 SP1-plus or 2008 R2, running on a server with at least 4GB of memory. Your database collation must be set to SQL-Latin1_General_CP1_CI_AS with full text search enabled.
In larger environments, you’ll want to cluster the SQL Servers for HA. Note that the data warehouse database is no longer optional, as it was in SCOM 2007 R2. It’s now a required component in your SCOM environment. You can, however, have it shared between management groups. The Windows agent comes in a 32- and a 64-bit version, as well as a 64-bit Itanium version.
Upgrade Path
You can only upgrade directly to SCOM 2012 from SCOM 2007 R2. All SCOM 2007 R2 management servers you want to upgrade must be 64-bit on x64 hardware and be running Windows Server 2008 R2 SP1.
The general order of upgrading is secondary management servers, gateways and agents first, then the Root Management Server (RMS). The upgrade will be blocked if any management servers or gateways are still SCOM 2007 R2. It will be highlighted during the RMS upgrade if any agents are still SCOM 2007 R2, but it won’t block the upgrade. Those agents just won’t be able to report until they’ve been upgraded to SCOM 2012 agents.
If you’re in a smaller environment with an all-in-one SCOM 2007 R2 server, you can either upgrade in place (provided your server meets the hardware and software requirements) or set up another management server and start the upgrade from there. If you go with the first option, you have to upgrade all the agents before they’ll report to SCOM 2012.
To help with migration planning, there are some excellent flow diagrams in the TechNet Library that clarify your options. Also provided are links to checklists with step-by-step instructions, in addition to an upgrade helper management pack (MP).
Some recommendations in planning your upgrade include backing up the databases, disabling notifications to prevent false alarms, stopping connectors to avoid false tickets being generated and ensuring agents aren’t reporting directly to the RMS during your upgrade. Above all, check the event log for any problems. You can’t upgrade your way out of a problem, so make sure your management groups are healthy before you upgrade.
You might have a blend of SCOM 2007 R2 and SCOM 2012 management groups and servers in your environment for a while, so it’s good to know SCOM 2012 agents will communicate with SCOM 2007 R2 servers. It won’t work the other way around, however, so it’s important to upgrade your older agents as soon as possible.
All MPs that work with SCOM 2007 R2 should work in SCOM 2012, because the MP schema is unchanged. Some exceptions include some third-party MPs requiring new modules on the agent, new MP templates or new view types due to API changes. Network monitoring MPs leveraging Simple Network Management Protocol (SNMP) will continue to work, but might require updating to integrate with the new network monitoring framework.
Infrastructure Improvements
In SCOM 2007 R2, the RMS is a single point of failure. It’s the connection point for consoles and any Web consoles. It runs the configuration service and handles connectors, health aggregation and role-based access control (RBAC). The only way to ensure HA in SCOM 2007 R2 is to cluster the RMS server. This can be technically and operationally complex. It also relies on an active/passive model with the associated hardware and licensing costs.
SCOM 2012 changes the game by taking the route of Microsoft Exchange and other Microsoft applications. HA is built in, right out of the box. No management server is more important than any other. Simply by having several in a pool, the load is balanced and availability is ensured. Each server runs the configuration service and stores data in the database, instead of in XML configuration files or memory like SCOM 2007 R2. This leads to quicker management server start-up.
Failover isn’t instantaneous. It can take up to two minutes while the pool reloads managed instances. Also, all management servers are treated as having equal capacity. Differences in processors and memory capacity aren’t taken into account. There are three default pools: the All Management Server Resource Pool, a Notification Pool and an Active Directory Integration pool. You can also create your own pools for specific needs.
Some MPs (such as those for Exchange Server 2007 and Exchange Server 2010) rely on an RMS. Because there isn’t an RMS server in SCOM 2012, one management server is assigned the RMS Emulator role to provide compatibility with these MPs. You can manually move this role between management servers, but there’s an MP coming that will automate role failover.
You can manually control roles within a pool. This is suitable if you have a hardware text/SMS alerting device connected to a particular management server. There’s no point in failing that function over to another server that doesn’t have attached hardware.
Interoperability Issues
One issue with the current System Center suite is that it’s essentially different applications with little integration. That’s about to change with the 2012 version. The glue that ties these different programs together is System Center Orchestrator 2012. It provides Integration Packs (IPs) for each of the major System Center applications, including SCOM. The SCOM IP can create and interact with Alerts and Monitors, as well as start and stop maintenance mode.
There are also IPs for System Center Service Manager (SCSM). These can automatically create incidents based on alerts in SCOM. For example, the IP for System Center Virtual Machine Manager (VMM) can push information about VMs, services, private clouds and hosts into SCOM. It will be interesting to see if this new approach to integrating the System Center suite will finally provide the integration glue asked for by so many.
The current SCOM 2007 R2 uses connectors to integrate with other management systems like IBM Tivoli, HP OpenView and others. SCOM 2012 doesn’t support these. Integration between SCOM and other management systems will be facilitated through System Center Orchestrator 2012.
Windows PowerShell Support
The good news is that SCOM 2012 comes with full Windows PowerShell 2.0 support and a host of new cmdlets. There will be a learning curve, as the new cmdlet nouns have “SCOM” in their names. The old cmdlets still seem to work, though. There are also new cmdlets for monitoring Unix and Linux machines. These rely on Windows PowerShell 3.0, which is currently in Community Technology Preview (CTP).
To execute Windows PowerShell cmdlets, you’ll have to establish a connection to a management group. You can make this a persistent connection (so you can run multiple cmdlets), or temporary to run a single command.
Advanced Monitoring
In a larger enterprise, network troubleshooting and maintaining servers are often two separate jobs. This makes it difficult to quickly determine if a problem is with the hardware, the OS or the network. One of the most exciting additions to SCOM 2012 is network monitoring. This is specifically designed to increase visibility and help you solve problems more quickly.
SCOM 2007 R2 offers basic network device monitoring, but not at the port level (unless you manually configure each device). SCOM 2012 supports SNMP v1, v2 and v3, and works with both IP v4 and v6. The new SNMP stack is native to SCOM 2012. SCOM 2007 R2 used the OS SNMP stack.
You’ll be able to monitor any network device that responds to SNMP down to the port level. You can also exclude particular devices from discovery. There’s also extended monitoring that tracks processor and memory utilization/fragmentation, along with other device-specific items for devices supported by SCOM 2012. To date, there are more than 80 vendors and 800 devices supported.
If a device supports SNMTP traps for system changes (such as a card added or changes to the chassis configuration), SCOM 2012 listens for these. Read Only SNMP strings do all the monitoring. If a node is down, all other monitoring is suppressed so you’re not flooded with alerts about ports and links being down (see Figure 2).
Figure 2 Network monitoring data can help you quickly spot problems.
The port-stitching feature shows which agent-monitored node is connected to each port. SCOM will also discover VLANs and the switches participating in each VLAN. It will only monitor connected ports, unless you manually add ports to the critical network port rule. For Cisco routers, it identifies the Hot Standby Router Protocol (HSRP) groups in which they participate. SCOM 2012 has more than 200 new items for network monitoring.
Currently, the recommended scalability numbers for SCOM 2012 are 500 devices per management server and 2,000 devices per management group. A more comprehensive sizing guide is forthcoming. You can only have one discovery rule per management server, so make sure it encompasses all the devices you need to find.
There are four dashboards for network monitoring. The Network Vicinity Dashboard provides a visual representation of connected devices within one hop to the selected node. You can increase the number of hops up to five. This dashboard won’t identify teamed NICs as such, nor will it show Unix/Linux computers. VMs will be associated with the same network device as the host, although the Hyper-V switch does show up as an SNMP device. The other monitoring dashboards give you an overview of problem devices, specific information about particular devices and individual port information.
Troubleshooting Tactics
Troubleshooting application performance issues is a difficult area. It often requires intimate knowledge of a particular program’s workings. Is the problem in the code, the server hardware, the server software or the network? You need standard metrics across all applications and a way to easily pinpoint the tier in which the problem might lie.
AVIcode looks for performance problems in application code. It was recently acquired by Microsoft and will be integrated into SCOM as Application Performance Monitoring (APM). APM will only work with Microsoft .NET Framework/Web applications, not standalone executables, and it only monitors IIS 7 or 7.5.
The SCOM 2012 infrastructure is totally integrated. There’s no separate database. If it’s monitoring a Windows Server 2008/2008 R2 machine running IIS, it will automatically deploy the APM agent. You can also set an overall Service Level Agreement, or SLA, (defining how many seconds wait is unacceptable) for all Web applications, rather than having to configure monitoring for each individually. You can then tweak the SLA for particular programs as needed.
When the interceptors are activated and loaded into IIS, the server will require a restart. After that, even if you add additional applications, you only have to recycle the particular app pool (see Figure 3).
Figure 3 Configuring Application Performance Monitoring isn’t difficult—you should be able to easily monitor applications.
The beauty of that level of integration becomes apparent when you see network, hardware and OS monitoring right next to the application performance information. This makes it much easier to zero in on the problem.
Dealing with Dashboards
SCOM collects vast amounts of data. It’s not a matter of collecting the data, though. It’s a matter of filtering and displaying the right data to the right people at the right time. Graphs and charts are a great way of doing this. Earlier versions of SCOM had views and simple dashboards, but SCOM 2012 takes it to a whole new level.
The new wizard for creating dashboards makes it easy to display data customized for particular audiences. The wizard is available in both the native and Web consoles. You can display the resulting dashboards in the Console, the Web Console and SharePoint 2010. They look nearly identical in all three environments. SCOM 2012 can have nested dashboards, where drilling down into particular data leads to another dashboard.
There are three steps to creating a dashboard in SCOM 2012. First, select a layout based on the number of cells desired. Add a widget in each cell (widget types include Alert, Performance and State). Finally, configure each widget with scope, criteria and display preferences (see Figure 4).
Figure 4 Creating custom dashboards in System Center Operations Manager 2012 is a simple process.
You can now integrate SCOM dashboards into SharePoint 2010, using a Web part. If the people who are going to view the dashboards aren’t SCOM users, you can configure a Web part with shared credentials. The integration works with SharePoint Server 2010 Standard and Enterprise, as well as the free Foundation version.
Any dashboard personalization is now stored in the database, so it can follow you to different PCs and environments. In SCOM 2007 R2 the dashboards were stored in the registry on the local machine. Dashboards in the Web console all have a distinct URL. This makes it easy to disseminate information to non-IT users because they can simply bookmark particular dashboards.
The most popular built-in dashboard might be the new Management Group Health Dashboard console, also known as the “coffee break.” The team calls it that because it’s designed to give SCOM 2012 operators a quick overview of their environment, thus answering the question, “Can I take a coffee break?” It monitors both the infrastructure and the functions delivered by the SCOM system (see Figure 5).
Figure 5 The Management Group Health Dashboard gives you a quick rundown.
Mixed-Environment Monitoring
SCOM 2012 continues native support for monitoring Unix and Linux (*nix) machines, with some important improvements. The *nix agent supports HP-UX 11i version 2 or 3 on PA-RISC and IA64; Sun Solaris 9 on SPARC and 10 on SPARC and x86; Red Hat Enterprise Linux 4, 5 and 6 on both x86 and x64; Novell SuSE Linux Enterprise Server 9 on x86, 10 SP1 and 11 on both x86 and x64; and IBM AIX 5.3, 6.1 and 7.1 on POWER.
To facilitate mixed-environment monitoring, SCOM 2012 supports sudo and SSH keys. The former means you can configure a standard account on managed machines with exactly the required amount of permissions. The latter ensures that agent maintenance is secure.
SCOM 2012 also brings comprehensive support for monitoring Java Enterprise Edition (JEE, formerly known as J2E) application servers. It supports four servers: IBM WebSphere 6.1 and 7; Red Hat JBoss 4.2, 5.1 and 6; Oracle WebLogic 10g Rel3 and 11g Rel1; and the open source Apache Tomcat 5.5, 6 and 7 on both Windows and Linux. It also supports WebSphere on AIX and WebLogic on Solaris.
After importing the Java MPs to match your environment, it should automatically discover the application servers. Standard monitoring lets you know if the application server is running and if resource utilization is within defined thresholds.
For deeper monitoring, Microsoft has an open source Java Management Extension (JMX) application called BeanSpy (previously known as JMX Extender) that you load on the application server. It reports to SCOM via either HTTP or HTTPS, with or without basic authentication. BeanSpy communicates with MBean counters (a bit like performance counters in Windows) to monitor individual applications running, frequency and time spent on memory garbage collection, as well as performance of the application server.
It’s disappointing that SCOM has no native support for Windows clustering. It would also be good to see application monitoring of Windows Azure public cloud applications, as well as clustered Java applications in JEE.
Overall, SCOM 2012 is a thorough revamp with some very useful new features. The simplified infrastructure and no-brainer HA will be welcome. The advanced network monitoring should make your troubleshooting life easier. Perhaps the most intriguing feature, though, will be seeing how System Center Orchestrator will glue the entire System Center suite together.
Paul Schnackenburg has been working in IT since the days of 286 computers. He works part time as an IT teacher as well as running his own business, Expert IT Solutions, on the
Sunshine Coast of Australia. He has MCSE, MCT, MCTS and MCITP certifications and specializes in Windows Server, Hyper-V and Exchange solutions for businesses. Reach him at paul@expertitsolutions.com.au and follow his blog at TellITasITis.com.au.