HealthServiceWatcher alerts and connector.

Recently I had heard from couple of customers (internal and external) about their trouble with ticketing alerts created from Health Service Watcher. Main complaint was related to the fact that alert is raised by instance of the watcher rather than instance of the computer that experiences a problem as well as the fact that FQDN of computer in question is buried too deep inside of context causing direct notification of computer owner becomes impossible.

Original heart beat missing alert

In this post, I do not want to justify or argue our design, but I would like to offer possible solution to this problem. And as we always recommend customization of alert generated by monitor state change thru connector, I decided to try and implement this solution thru the connector myself. So I ventured into (to me unknown) world of managed coding and OpsMgr SDK, and created my first connector for OpsMgr 2007.

Hooray! It only took me a weekend of work, but on the other hand, I’m a lucky one as I have access to source code. (I personally do not believe that documentation of this feature is sufficient as except couple of references, I could not really find any deeper HOWTOs).

Once I choose to implement OpsMgr connector, I had couple of options. I could have subscribed to HSW alerts and customize its properties while changing CustomFieldX values. That I found somewhat trivial so decided to try something else instead.

I wondered the path of creating brand new alert, where this new alert is in fact created by the instance of the computer which experienced the problem.

NOTE: It is not known, but since the beginning of times, there is module which is able to generate alert for different instance of different type than target instance of the workflow which loaded this module. It is somewhat tricky and not the purpose of this post to explain such module, so I will leave this to those who are trying to reverse engineer to figure out how it works (at least for now).

It also seem to be a requirement of connector subscription to have a singleton group populated with instances of ME type for which you want to subscribe to alert forwarding, so I created such group and populated its instance with instances of HealthServiceWatcher.

NOTE: To do all this, I leveraged a feature of OpsMgr SDK and made my MP a part of the connector. This MP is imported while initializing connector and will be removed when connector is removed. You can export this MP and investigate its functionality (in fact I attached it to this post) but you must keep it present for my connector to work (its removal will cause connector termination). For those who are C# skilled, this MP is implemented inside of MPCreator.cs file (attached as well).

Here is the approach I chosen for my implementation. Subscription is using 1 minute polling interval to recognize which HSW alerts should be forwarded. Again, forwarded alerts are from instances that are contained within “Health Service Watcher Group For Connector” (see connector subscription and section Groups for additional details).

Connector properties

Connector is also using 1 minute timer callback to request forwarded alerts and takes the action based on the resolution state. When HSW alert is active, its resolution state is changed to custom value (1) and SDK event is inserted into OpsMgr DB. Rule (which is again defined inside of custom MP) is generating computer alert and is heavily reusing information from original alert (as such was part of <Data> section in SDK event). After this, alert raised for computer can beeasily used with ticketing system. When HSW alert is resolved, alert created for computer is resolved automatically as well. Reason is because connector is notified about HSW resolution state change.

NOTE: There is an issue with alert “duplicity”. Although connector has access to HSW alert, such cannot be resolved by connector while it is processed. Main reason for that is the fact, that “healthy” monitor state change is resolving this alert, but such is no longer causing subscription to pick and forward this alert to the connector because ultimately it has not changed from subscription point of view and I wanted to simulate auto-resolve feature as original alert is created by HSW monitor state change. This also means that manually resolving HSW alert while in console causes automatic resolution of computer alert, as connector is notified and performs close computer alert action automatically.

New resolution for heart beat missing alert

Alert generated for computer when heartbeat missing.

RECAP:

1. Place HealthServiceWatcherConnector.exe from attached compressed file to your RMS

2. Initialize connector after you start/execute application. This action will import management pack, creates connector and its subscription and starts timer to process forwarded alerts.

3. Use “Manual Refresh” outside of regular timer callback to “force” alert processing.

4. Exiting application will preserve connector registration, its subscription as well as MP imported. Next start, initializing connector only means to activate subscription.

5. Removing connector will unregister connector, deletes MP and cleans system.

 

NOTE: There is a scalability issue that is worth to mention. Initial connector processing will need to process all HSW alerts that were ever raised, so expect CPU spikes. Also disconnecting RMS from network causes all computers in management group to report its unavailability which causes alert flood and possible CPU spikes for connector process again.

 

DISCLAIMER:

Please evaluate in your test environment first! As expected, this solution is provided AS-IS, with no warranties and confers no rights. Use is subject to the terms specified at Microsoft.

HealthServiceWatcherConnector.zip