·         Query for all classes that match a given class name.

·         Query for all relationships with a particular target class.

·         Query for all tasks with a particular category.

·         Query for all management packs of a particular version and so on.

There is a criteria class associated with each of these types. Example, the class  ManagementPackClassCriteria is used to query for management pack classes.

namespace Microsoft.EnterpriseManagement.Configuration

public class ManagementPackClassCriteria : QueryCriteria<ManagementPackClassCriteria>

    {

    }

The following table indicates all the properties of a management pack class that can be queried for. Example – All classes with a specific management pack id, classes that are abstract, classes that are sealed etc.

The constructor for these classes accepts a criteria xml as shown below:

public ManagementPackCriteria(string criteria)

{

}                             

All type related criteria classes provide a similar constructor. The criteria parameter follows the same schema as discussed in this post except for the following differences:

·         Only <GenericProperty> tags are used and not <Property> for indicating properties to query. As a result the syntax followed is <GenericProperty>propertyName</GenericProperty>

·         ContainmentCriteriaType and FullTextCriteriaType cannot be used.

·         Guid values for types need to be used instead of type names as there is no managementpack parameter exposed and hence the type name cannot be resolved.

Here’s an example of a criteria xml that queries for all classes in a given management pack.

<Criteria xmlns="https://Microsoft.EnterpriseManagement.Core.Criteria/">

  <Expression>

    <SimpleExpression>

      <ValueExpressionLeft>

        <GenericProperty>ManagementPackId</GenericProperty>

      </ValueExpressionLeft>

      <Operator>Equal</Operator>

      <ValueExpressionRight>

        <Value>dbb151b7-53e0-4de0-967e-a1ade70f5c3c</Value>

      </ValueExpressionRight>

    </SimpleExpression>

  </Expression>

</Criteria>

The code for the same looks as below:

ManagementPackClassCriteria criteria = new ManagementPackClassCriteria(criteria);

IList<ManagementPackClass> mpClasses = managementGroup.EntityTypes.GetClasses(criteria);

In this post, we talked about querying for all incidents that were related to config items. Consider querying for all relationships that exist in the system that include config item as a source or target. The criteria xml for the same would look like:

<Criteria xmlns="https://Microsoft.EnterpriseManagement.Core.Criteria/">

  <Expression>

    <Or>

      <Expression>

        <SimpleExpression>

          <ValueExpressionLeft>

            <GenericProperty>SourceType</GenericProperty>

          </ValueExpressionLeft>

          <Operator>Equal</Operator>

          <ValueExpressionRight>

            <Value>replace with guid value for config item type</Value>

          </ValueExpressionRight>

        </SimpleExpression>

      </Expression>

      <Expression>

        <SimpleExpression>

          <ValueExpressionLeft>

            <GenericProperty>TargetType</GenericProperty>

          </ValueExpressionLeft>

          <Operator>Equal</Operator>

          <ValueExpressionRight>

            <Value>replace with guid value for config item type</Value>

          </ValueExpressionRight>

        </SimpleExpression>

      </Expression>

    </Or>

  </Expression>

</Criteria>

The following table indicates all the properties on which a ManagementPackRelationship can be queried for

Note: For operations that involve querying for classes or relationships frequently, it will be better to use regular APIs without criteria and compare results on the client for performance reasons.

     namespace Microsoft.EnterpriseManagement.Common

    public class EnterpriseManagementObjectCriteria : QueryCriteria<EnterpriseManagementObjectCriteria>, IObjectCriteria

    {

         }

    public class ObjectProjectionCriteria

    {

         }

Class EnterpriseManagementObjectCriteria works on pure instances. Say, you wanted to query for work items and were looking for all work items that satisfied a particular condition.

Class ObjectProjectionCriteria is used for querying over instances based on some condition that spans over a relationship. Say, you wanted to query for all incidents that were assigned to a certain user with some criteria acting on the relationship “IncidentAssignedToUser”.

Constructors of both these classes (EnterpriseManagementObjectCriteria and ObjectProjectionCriteria) work on an input criteria parameter which is a schema based XML string, the schema of which varies slightly between both the classes.

Most likely your query will fall into one of these categories which correspond to a specific type of schema element in your criteria XML.Please refer to more specific examples below for a given scenario. These descriptions and the examples that followed are explained with an assumption that readers have knowledge of core concepts such as MP definitions, type projections, relationships and various types of relationships such as reference or containment.

• You are querying over an instance (that exists by itself or in a relationship) evaluating a property on it for a specific value. (Please refer to  SimpleCriteriaType and CriteriaOperatorType in the schema)
• You are querying over an instance (that exists by itself or in a relationship) evaluating a property on it based on a regular expression. (Please refer to  RegExCriteriaType and RegexOperatorType in the schema) 
• You are querying over an instance (that exists by itself or in a relationship) evaluating a property in a similar fashion such as the SQL IN/NOT IN clause. (Please refer to type InType/NotInType in the schema)
• You want to check if an instance actually exists and not are necessarily filtering on a particular value. (Please refer to type UnaryCriteriaType and UnaryOperatorType in the schema)
• Your query spans over some form of Containment relationship. (Please refer to  ContainsCriteriaType,NotContainsCriteriaType, ContainedCriteriaType, NotContainedCriteriaType in the schema)
• You are querying over an instance (that exists by itself or in a relationship) evaluating a property on it based on a full text expression. (Please refer to  FullTextCriteriaType and FullTextOperatorType in the schema)
• You have a NOT expression in your criteria (Please refer to NotType Criteria).
• Your criteria involves performing a combination of the above functions. (Please refer to And Type, Or Type in the schema)

The combination of all examples stated below will give an understanding of some core concepts in criteria.

Example 1: Querying for an instance on some property using simple expression criteria, using EnterpriseManagementObjectCriteria class.

Consider a query for retrieving all instances of a certain class based on some property value. Say, fetching all incidents whose title contains the text “critical”. Since this is purely evaluating a particular property on the criteria we can construct an SDK criteria of type “SimpleCriteriaType” for the purpose. As per the schema all criteria start with an “ExpressionType”

All property evaluation is of the format: Context/ [Type]/ [PropertyName] (All start with a Context tag which is required and denotes the type of instance we are querying for, [Type] is replaced by the name of the type we are querying on, and in this case “System.WorkItem.Incident” and [PropertyName] is the name of the property which in this case is “Title”.

We replace the inner text of the <Value> tag with the actual value to be compared with. Since we are performing a Like operation the value is appended with % as in %critical%.

This forms criteria Xml as below:

<Criteria xmlns="https://Microsoft.EnterpriseManagement.Core.Criteria/">

  <Expression>

    <SimpleExpression>

      <ValueExpressionLeft>

        <Property>$Context/Property[Type='System.WorkItem.Incident']/Title$</Property>

      </ValueExpressionLeft>

      <Operator>Like</Operator>

      <ValueExpressionRight>

        <Value>%critical%</Value>

      </ValueExpressionRight>

    </SimpleExpression>

  </Expression>

</Criteria>

This piece of code creates an EnterpriseManagementObjectCriteria object:

ManagementPack incidentManagementPack = managementGroup.GetManagementPack("System.WorkItem.Incident.Library", "31bf3856ad364e35", “1.0.0.0”);

ManagementPackClass incident = managementGroup.EntityTypes.GetClass("System.WorkItem.Incident", incidentManagementPack);

EnterpriseManagementObjectCriteria criteria = new EnterpriseManagementObjectCriteria(criteriaXml,incident, incidentManagementPack,managementGroup);

(CriteriaXml here indicates the xml based criteria string above)

You could also opt to use the constructor below, however since the management pack is excluded, the string criteria has to include the guid values for the class names and property names instead of the name.

EnterpriseManagementObjectCriteria criteria = new EnterpriseManagementObjectCriteria(criteriaXml, incident,managementGroup);

The same criteria with guid values for the type and property will look as shown below:

<Criteria xmlns="https://Microsoft.EnterpriseManagement.Core.Criteria/">

  <Expression>

    <SimpleExpression>

      <ValueExpressionLeft>

        <Property>$Context/Property[Type=8C61EEAB-9F24-0EB5-5B4A-BF4DD59C0EF3']/ 0C7DC36E-876F-8B2B-C53B-A6297208E6F0$</Property>

      </ValueExpressionLeft>

      <Operator>Like</Operator>

      <ValueExpressionRight>

        <Value>%critical%</Value>

      </ValueExpressionRight>

    </SimpleExpression>

  </Expression>

</Criteria>

Example 2: Querying on related instances using Object Projection Criteria and usage of unary expression

The above example talks about retrieving instances of a particular class, however say you wanted to retrieve those instances based on a certain relation. Consider querying for all incidents that were related to a config item. These are instances that are part of the relationship “System.WorkItemRelatesToConfigItem”. Note that we need to query over this relationship here.  Also, we are looking for incidents that are related to any config items that exist and not necessarily a config item of a particular property value. Hence we will be using “UnaryExpression” criteria in this case.

As per the schema the unary expression criteria looks like below:

<Criteria xmlns="https://Microsoft.EnterpriseManagement.Core.Criteria/">

  <Expression>

    <UnaryExpression>

      <ValueExpression>

        <GenericProperty Path="$Context/Path[Relationship='WorkItem!System.WorkItemRelatesToConfigItem']$">Id</GenericProperty>

      </ValueExpression>

      <Operator>IsNotNull</Operator>

    </UnaryExpression>

  </Expression>

</Criteria>

Unary expressions contain just an expression and an operator to check for null. Notice how we are using a generic property tag instead of a property tag here. This is used for querying over generic properties that are common to all instances and not just properties specific to that particular instance. Following are the list of generic properties that we can be used in this format.

public enum EnterpriseManagementObjectGenericPropertyName

       {

        Id,

        Name,

        Path,

        FullName,

        DisplayName,

        LastModified,

        TimeAdded,

        LastModifiedBy

       }

As long as the property is one of the above we can use a <GenericProperty> tag and need not specify the type in the criteria. The attribute Path indicates the relationship on which we are using the criteria. The Path usually starts with “Context/” and is followed by one or more relationships. We have just one in this case. You can use the above sdk criteria to build an Object Projection Criteria object.

Add a projection that defines the relationship - System.WorkItemRelatesToConfigItem in a custom management pack as below:

<TypeProjection ID="WiRelatesToCiProj" Accessibility="Public" Type="CoreIncident!System.WorkItem.Incident">

  <Component Path="$Target/Path[Relationship='WorkItem!System.WorkItemRelatesToConfigItem']$" Alias="Rel" />

</TypeProjection>

The following code creates an ObjectProjectionCriteria :

ManagementPack workItemMp = managementGroup.GetManagementPack("System.WorkItem.Library", "31bf3856ad364e35", new Version("1.0.0.0"));

ManagementPack projMp = managementGroup.GetManagementPack("custom.Mp",null, new Version("1.0.0.0"));

ManagementPackTypeProjection workItemRelConfigProj =managementGroup.EntityTypes.GetTypeProjection("System.WorkItem.Incident.RelatesToConfig", projMp);

ObjectProjectionCriteria projCriteria = new ObjectProjectionCriteria(criteriaxml, workItemRelConfigProj, projMp, managementGroup);

    (criteria xml is the above mentioned xml string).

Just like EnterpriseManagementObjectCtieria you can also use the constructor that does not take a management pack, but will need to specify guids for all elements.

Example 3: Using typeconstraints, references and tokens in criteria. Usage of And/Or criteria.

Consider the scenario where you are not only querying for workitems related to config items but for those that are related to a specific config item based on some property. Here’s where using a type constraint is useful. Consider the following criteria.

<Criteria xmlns="https://Microsoft.EnterpriseManagement.Core.Criteria/">

  <Reference Id="System.Library" Version="1.0.0.0" Alias="System" />

  <Expression>

    <And>

      <Expression>

        <SimpleExpression>

          <ValueExpressionLeft>

            <Property>$Context/Path[Relationship='System.WorkItemRelatesToConfigItem' TypeConstraint='System!System.Computer']/Property[Type='System!System.Computer']/DisplayName$</Property>

          </ValueExpressionLeft>

          <Operator>Equal</Operator>

          <ValueExpressionRight>

            <Value>testmachine</Value>

          </ValueExpressionRight>

        </SimpleExpression>

      </Expression>

      <Expression>

        <SimpleExpression>

          <ValueExpressionLeft>

            <GenericProperty Path="$Context/Path[Relationship='System.WorkItemAssignedToUser' SeedRole='Source']$">Id</GenericProperty>

          </ValueExpressionLeft>

          <Operator>Equal</Operator>

          <ValueExpressionRight>

            <Token>[me]</Token>

          </ValueExpressionRight>

        </SimpleExpression>

      </Expression>

    </And>

  </Expression>

</Criteria>

Note :

Ø  The typeconstraint restricts the filter to just config items of type System.Computer.

Ø  The type “System.Computer” is referred with an alias “System!” since it’s defined in a different management pack. The reference is added to the existing MP.

Ø  The criteria combines two different expression in an “AND” condition and hence queries for all workitems that are assigned to me (current user) and that also have a relationship with class “System.Computer”.

Ø  The token [me] instead of the value to be compared with resolves to the current user. Other tokens that are allowed are :

[me] – denotes current user

[mygroups] – denotes current user’s group

[now] – denotes the current utc time.

[now + 5s], [now – 2h] – denotes some variations using s(seconds), d (days), m(months), h(hour).

Example 4: Using sdk criteria to create customizable views and usage of Containment criteria.

Say you have a containment relationship that you want to query over. Consider a containment relationship where you want to group all incidents related to hardware under a group “Hardware Incidents”.

Currently the SM console, does not offer a way for doing the same.

However, we can achieve the same by adding SDK criteria and customize the view.

Step 1) In the SM console, go to WorkItems -> IncidentManagement.

Step 2) Under Tasks choose “Create View”

Step 3) Edit the General and Display tabs, do not add any criteria under the Criteria tab. Enter view name as “Hardware Related Incidents”.

Step 4) Save the view. The view will show up under incident management and get saved in the unsealed management pack “Service Manager Incident Management Configuration Library”.

Step 5) Export this MP.

Step 6) Add the following class, relationship and type projection to the MP.

<ClassType ID="IncidentHardwareGroup" Base="WorkItem!System.WorkItemGroup" Abstract="false" Accessibility="Public" Hosted="false">

  <Property ID="GroupId" Type="string" Key="true"  />

</ClassType>

<RelationshipType ID="IncidentGroupContainsIncidents" Base="System!System.Containment" Abstract="false" Accessibility="Public">

  <Source ID="incidentGrp" Type=" IncidentHardwareGroup " />

  <Target ID="incident" Type="CoreIncident!System.WorkItem.Incident" />

</RelationshipType>

<TypeProjection ID="IncidentGrpProj" Accessibility="Public" Type="CoreIncident!System.WorkItem.Incident">

  <Component Path="$Target/Path[Relationship=' IncidentGroupContainsIncidents ' SeedRole='Target']$" Alias="group" />

</TypeProjection>

Step 7) Under the definition generated for this view you will notice an empty criteria tag (<Criteria/>), since we did not pick a criteria while creating this view.

Step 8) Replace <Criteria/> with the criteria below.

<Criteria>

  <QueryCriteria Adapter="omsdk://Adapters/Criteria" xmlns="https://tempuri.org/Criteria.xsd">

    <Criteria>

      <FreeformCriteria>

        <Freeform>

          <Criteria xmlns="https://Microsoft.EnterpriseManagement.Core.Criteria/">

            <Expression>

                  <Contained>

                      <Class>$MPElement[Name='System.IncidentCriticalGroup']$</Class>

              </Contained>

               </Expression>

               </Criteria>

        </Freeform>

      </FreeformCriteria>

    </Criteria>

  </QueryCriteria>

</Criteria>

The highlighted portion is the actual sdk criteria, rest of the format is needed for UI purposes. If you wanted to have some criteria on the group as well, you could have a criteria such as:

<Criteria xmlns="https://Microsoft.EnterpriseManagement.Core.Criteria/">

   <Expression>

     <Contained>

      <Class>$MPElement[Name='System.IncidentCriticalGroup']$</Class>

       <Expression>

         <SimpleExpression>

           <ValueExpressionLeft>

            <Property>$Context/Property[Type='System.IncidentCriticalGroup']/GroupId$</Property>

          </ValueExpressionLeft>

          <Operator>Equal</Operator>

           <ValueExpressionRight>

            <Value>HardwareRelated</Value>

          </ValueExpressionRight>

        </SimpleExpression>

      </Expression>

    </Contained>

  </Expression>

</Criteria>

Step 9) Change the name of the type projection in the view to “IncidentGrpProj”.

Step 10) Import the MP.

Step 11) The view will get updated and show the filtered instances as below.

I have covered the important concepts in SDK criteria and examples on querying over instances using the same. Using SDK criteria for querying over types will be covered in a different post. Hope you find this useful.

This is a continuation of my previous post on Microsoft.EnterpriseManagement.Common.ObjectQueryOptions. The previous post introduced the basics of object retrieval from the CMDB. In this post, I want to explain how you to get back results from the CMDB sorted in some order using ObjectQueryOptions.

Objects of a class or projection can be retrieved from the CMDB sorted by properties of the object.

There are two kinds of properties in every object. Those that are defined on a class in a management pack and those that are applicable to all objects for e.g LastModified, TimeAdded etc..

The properties applicable to all objects are called generic properties. The entire list of generic properties is below:

namespace: Microsoft.EnterpriseManagement.Configuration

public enum.EnterpriseManagementObjectGenericPropertyName

{

        Id = 0,

        Name = 1,

        Path = 2,

        FullName = 3,

        DisplayName = 4,

        LastModified = 5,

        TimeAdded = 6,

        LastModifiedBy = 7

}

Case 1: Sorting objects of a class

The relevant API on ObjectQueryOptions  for sorting class objects

/// <summary>

/// Used for sorting isntances by property

/// </summary>

/// <param name="property">Property to sort on</param>

/// <param name="sortOrder">Sort Order</param>

public void AddSortProperty(ISortableProperty property, SortingOrder sortOrder)

The above API is used to specify the sorting on objects of a class retrieved from the CMDB.

The property parameter can of type

 Microsoft.EnterpriseManagement.Configuration. ManagementPackProperty

Or

Microsoft.EnterpriseManagement.Configuration.EnterpriseManagementObjectGenericProperty

Let’s look at an example of retrieving all available Microsoft.Windows.Computer from the CMDB sorted by ascending order of property NetworkName (defined on class Microsoft.Windows.Computer) and ascending order of generic property TimeAdded

EnterpriseManagementGroup emg = new EnterpriseManagementGroup("sm-server");

ManagementPackClass mp = emg.ManagementPacks.GetManagementPack("Microsoft.Windows.Library", keyToken, new Version("7.5.1049.1");

ManagementPackClass computer = emg.EntityTypes.GetClass("Microsoft.Windows.Computer", mp);

ObjectQueryOptions queryOptions = new ObjectQueryOptions(ObjectPropertyRetrievalBehavior.All);

// sort based on NetworkName

ManagementPackProperty networkName = computer["NetworkName"];

queryOptions.AddSortProperty(networkName, SortingOrder.Ascending;

// sort based on LastModified

EnterpriseManagementObjectGenericProperty genericProperty =

new EnterpriseManagementObjectGenericProperty(EnterpriseManagementObjectGenericPropertyName.TimeAdded);

queryOptions.AddSortProperty(genericProperty, SortingOrder.Ascending);

IObjectReader<EnterpriseManagementObject> instanceReader = emg.EntityObjects.GetObjectReader<EnterpriseManagementObject>(computer, queryOptions);

Case 2: Sorting objects of projection types.

The relevant APIs on ObjectQueryOptions  for sorting projection objects

// <summary>

/// Projection sorting based on generic and non-generic properties

/// </summary>

/// <param name="sortXml">properties to sort on, specified in xml</param>

/// <param name="projection">the projection type</param>

/// <param name="managementPack">the management pack to be used for resolving the properties specified in the sort xml</param>

/// <param name="managementGroup">the management group</param>

public void AddSortProperty(string sortXml, ManagementPackTypeProjection projection, ManagementPack managementPack, EnterpriseManagementGroup managementGroup)

// <summary>

/// Projection sorting based on generic and non-generic properties

/// </summary>

/// <param name="sortXml">properties to sort on, specified in xml</param>

/// <param name="projection">the projection type</param>

/// <param name="managementGroup">the management group</param>

public void AddSortProperty(string sortXml, ManagementPackTypeProjection typeProjection, EnterpriseManagementGroup managementGroup)

Below is an example of sorting objects of type System.WorkItem.Incident.ProjectionType.

Let’s retrieve all incidents assigned to user “Peter”, by descending order of priority (property defined on  class System.WorkItem.TroubleTicket) and ascending order of AffectedUser (defined on class  System.Domain.User )name in each incident.

Here, is the criteria I used for getting incidents assigned to “Peter”

<Criteria xmlns="https://Microsoft.EnterpriseManagement.Core.Criteria/">

  <Reference Id="System.WorkItem.Library" Version="7.5.1083.0" Alias="WorkItem" PublicKeyToken="9396306c2be7fcc4"/>

  <Reference Id="System.Library" Version="7.5.1083.0" Alias="System" PublicKeyToken="9396306c2be7fcc4"/>

  <Expression>

    <SimpleExpression>

      <ValueExpressionLeft>

       <Property>$Context/Path[Relationship='WorkItem!System.WorkItemAssignedToUser' TypeConstraint='System!System.Domain.User']/Property[Type='System!System.Domain.User']/UserName$</Property>

      </ValueExpressionLeft>

      <Operator>Equal</Operator>

      <ValueExpressionRight>

        <Value>Peter</Value>

      </ValueExpressionRight>

    </SimpleExpression>

  </Expression>

</Criteria>

Here is the sort xml I used:

<Sorting xmlns="https://Microsoft.EnterpriseManagement.Core.Sorting">

  <Reference Id="System.WorkItem.Library" Version="7.5.1083.0" Alias="WorkItem" PublicKeyToken="9396306c2be7fcc4"/>

  <Reference Id="System.Library" Version="7.5.1083.0" PublicKeyToken="9396306c2be7fcc4" Alias="System" />

  <SortProperty SortOrder="Descending">$Context/Property[Type='WorkItem!System.WorkItem.TroubleTicket']/Priority$</SortProperty>

  <SortProperty SortOrder="Ascending">$Context/Path[Relationship='WorkItem!System.WorkItemAffectedUser' TypeConstraint='System!System.Domain.User']/Property[Type='System!System.Domain.User']/UserName$</SortProperty>

</Sorting>

EnterpriseManagementGroup emg = new EnterpriseManagementGroup("sm-server");

ManagementPackClass mp = emg.ManagementPacks.GetManagementPack("ServiceManager.IncidentManagement.Library", keyToken, new Version("7.5.1083.0"));

ManagementPackTypeProjection typeProjection = emg.EntityTypes.GetTypeProjection("System.WorkItem.Incident.ProjectionType",mp);

ObjectProjectionCriteria criteria = new ObjectProjectionCriteria(Criteria.incident,typeProjection, emg);

ObjectQueryOptions opt = new ObjectQueryOptions();

opt.AddSortProperty(Criteria.sort,typeProjection, emg);

IObjectProjectionReader<EnterpriseManagementObject> p = emg.EntityObjects.GetObjectProjectionReader<EnterpriseManagementObject>(criteria, opt);

<Criteria xmlns="https://Microsoft.EnterpriseManagement.Core.Criteria/">
  <Expression>
    <SimpleExpression>
      <ValueExpressionLeft>
        <Property>$Context/Property[Type='System.WorkItem.Incident']/Classification$</Property>
      </ValueExpressionLeft>
      <Operator>Equal</Operator>
      <ValueExpressionRight>
        <Value>{9C46171F-64A2-B33A-FBD3-D3ED9881CDF6}</Value>
      </ValueExpressionRight>
    </SimpleExpression>
  </Expression>
</Criteria>  

In the above, "9C46171F-64A2-B33A-FBD3-D3ED9881CDF6" is the ID of list "IncidentClassificationEnum.Hardware". This criteria works as expected as long as "IncidentClassificationEnum.Hardware" does not have any child lists appended.  However, you may want to customize this list by adding more specific information, such as specifying what kind of hardware is related to incidents. Suppose this list hierarchy is changed as follows:

IncidentClassficationEnum.Hardware
|- IncidentClassficationEnum.Hardware.Disk
   |- IncidentClassficationEnum.Hardware.Disk.HardDisk
   |- IncidentClassficationEnum.Hardware.Disk.CD
|- IncidentClassficationEnum.Hardware.Memory
|- IncidentClassficationEnum.Hardware.CPU

Then you can create incidents and set the classification property as one of the above child lists.  In this case, in order to get all hardware related incidents, you need to adapt your query criteria.  In Service Manager 2010, your option was to use the 'In' operator:

<Criteria xmlns="https://Microsoft.EnterpriseManagement.Core.Criteria/">
  <Expression>
    <In>
      <Property>$Context/Property[Type='System.WorkItem.Incident']/Classification$</Property>
      <Values>
        <Value>9C46171F-64A2-B33A-FBD3-D3ED9881CDF6</Value>
        <Value>3D5430DC-EA64-C457-70D8-11D8257E0B58</Value>
        <Value>3C272B6C-A330-9135-E836-4E787EDE8C7C</Value>
        <Value>6185BF42-FBBA-4339-9AA9-DA0FDE48D881</Value>
        <Value>E1ECB59B-A9F0-9ED4-B675-B4C1C8CE97FD</Value>
        <Value>BCA04F7A-41E6-B053-6E40-EF4B02FDCCFD</Value>
      </Values>
    </In>
  </Expression>
</Criteria> 

In the above query, you have to collect all child list Ids as well as the parent list Id. Further, whenever you add more child lists, you need to change the criteria correspondingly.  In Service Manager 2010 SP1, we introduce a new operator "Under" in which you only need to specify the root list ID. The criteria keeps working if any child item is added, deleted or updated from the corresponding hierarchy.

<Criteria xmlns="https://Microsoft.EnterpriseManagement.Core.Criteria/">
  <Expression>
    <SimpleExpression>
      <ValueExpressionLeft>
        <Property>$Context/Property[Type='System.WorkItem.Incident']/Classification$</Property>
      </ValueExpressionLeft>
      <Operator>Under</Operator>
      <ValueExpressionRight>
        <Value>{9C46171F-64A2-B33A-FBD3-D3ED9881CDF6}</Value>
      </ValueExpressionRight>
    </SimpleExpression>
  </Expression>
</Criteria>

The under operator won't filter out the root list by default.  The results of using the above criteria include all incidents whose classification is equal to IncidentClassificationEnum.Hardware. If you only want to get incidents related to specific hardware, then you have to exclude the root list explicitly: 

 <Criteria xmlns="https://Microsoft.EnterpriseManagement.Core.Criteria/">
  <Expression>
    <And>
      <Expression>
        <SimpleExpression>
          <ValueExpressionLeft>
            <Property>$Context/Property[Type='System.WorkItem.Incident']/Classification$</Property>
          </ValueExpressionLeft>
          <Operator>NotEqual</Operator>
          <ValueExpressionRight>
            <Value>{9C46171F-64A2-B33A-FBD3-D3ED9881CDF6}</Value>
          </ValueExpressionRight>
        </SimpleExpression>
      </Expression>
      <Expression>
        <SimpleExpression>
          <ValueExpressionLeft>
            <Property>$Context/Property[Type='System.WorkItem.Incident']/Classification$</Property>
          </ValueExpressionLeft>
          <Operator>Under</Operator>
          <ValueExpressionRight>
            <Value>{9C46171F-64A2-B33A-FBD3-D3ED9881CDF6}</Value>
          </ValueExpressionRight>
        </SimpleExpression>
      </Expression>
    </And>
  </Expression>
</Criteria> 

This new operator has not been supported in UI criteria builder.  In order to use it in SM console, you need to do the changes manually.  Suppose you create a view to list all hardware incidents as shown below:

Notice that you cannot choose the 'Under' operator here.  Instead, select the 'equals' operator for now. Next, export the management pack where you saved this view and edit the view criteria to change 'Equal' operator to 'Under'.  After importing the updated management pack, click the above view and you are able to see all hardware related incidents there.

It is important to note that if you open the "edit view" dialog for the changed view, it still shows the 'Equals' operator in the criteria area.  This means that if you make any subsequent changes to the criteria of the view through the console, it will overwrite the 'Under' operator and you will have to manually edit the criteria again.

a.      How many objects are you expecting in the database for your criteria? How will the count affect your read performance?

b.      Do you want your results sorted?

c.      Do you want your results filtered by the source that created it?

d.      Do you want all your objects returned or only the TOP N results?

e.      Do you want all object properties returned or only some of them?

All Service Manager V1 APIs used for retrieving objects from the database take a parameter of type Microsoft.EnterpriseManagement.Common.ObjectQueryOptions. This class exposes several APIs that can be used to configure all of the above and more.

This is the first in a series of posts that I will be writing to go over each of the points above and see how we can use ObjectQueryOptions to achieve what we want.

Scenario 1: Large number of instances in the database; how to improve read performance

Suppose you have a large number of objects(1000+) in the CMDB that match your criteria; you may not want to or more importantly need to read the objects from the CMDB all at once, but instead retrieve only the first N objects and have the remaining objects load in the background (load on demand).

ObjectQueryOptions exposes a public property called ObjectRetrievalMode which is an enum that you can set. This property indicates whether you want the objects returned all at once (in a single batch) or in several small sized batches (aka pages)

/// <summary>

/// specifies whether the object retrieval from

/// server should be buffered or not

/// </summary>

public enum ObjectRetrievalOptions

{

    /// <summary>

    /// does not buffer the data

    /// </summary>

    NonBuffered,

    /// <summary>

    /// buffers the data

    /// </summary>

    Buffered

}

NonBuffered – you get all your objects returned in one batch.

Buffered – you’ll get your objects returned in batches ( you can specify the size of the batches as well). The default batch size is 250

Lets take a look at an example of how we can actually get back objects from the CMDB in batches. In the example below I am querying for  all of the objects of class Microsoft.Windows.Computer but accessing  only the first object.

Code snippet:

EnterpriseManagementGroup emg = new EnterpriseManagementGroup("sm-server");

ManagementPack mp = emg.ManagementPacks.GetManagementPack("Microsoft.Windows.Library", keyToken, new Version("7.5.1049.1");

ManagementPackClass computer = emg.EntityTypes.GetClass("Microsoft.Windows.Computer", mp);

ObjectQueryOptions queryOpt = new ObjectQueryOptions();

queryOpt.ObjectRetrievalMode = ObjectRetrievalOptions.Buffered;

IObjectReader<EnterpriseManagementObject> instanceReader = emg.EntityObjects.GetObjectReader<EnterpriseManagementObject>( computer, queryOpt);

// get the first object that matches the criteria

instanceReader.GetData(0);

NOTE: In the above example, even though I am accessing only the first object, the reader internally retrieves a the first page of instances from the CMDB.i.e the reader always retrieves enough instances from the CMDB to fulfill a client request, but at a minimum it gets back a page of instances from the CMDB.

In SMv1, the buffered/ non-buffered mode of retrieval can be used for retrieving objects and projections. The APIs return either IObjectReader<EnterpriseManagementObject> or IObjectProjectionReader<EnterpriseManagementObject> depending on whether instances or projection were queried for.

The batch or page size is the number of objects retrieved from the database at a given time. It can be set using the property on IObjectReader<T> or IObjectProjectionReader<T>.  The default page size is set to 250. It can be set to any acceptable value for an unsigned integer. If it is set to number greater than the actual number of instances matching a query, then only the available matching instances are brought back.

int PageSize

{

 get;

 set;

}

The instance and projection readers have many more powerful features that can be used to do a wide range of things to query for  instances from the CMDB. I will cover all the remaining features in a separate blogpost.

MSDN also now supports community contributions.  Please feel free to add your own comments, examples, code samples, etc!

https://msdn.microsoft.com/en-us/library/gg469708.aspx

I know that troubleshooting the data warehouse can be a bit intimidating. Sometimes even figuring out where to begin to look can be difficult. In this series of blog posts, I am going to help you figure out how to troubleshoot the data warehouse. I’ll use this post as an index with links to all of the subsequent topics.

Where to begin?

Depending on what kinds of issues you see, your start point might be different. Here I am going to talk about the general approach. 

If you observe a failure or something not too right, generally, there are a few places we can start our debugging.

1.      Start with the Operations Manager event log on the Data Warehouse server

This is the absolutely most important place to begin! Almost all of the errors from the Data Warehouse are output to this event log. Events in this log will have two different sources: Deployment and Data Warehouse.

Events with a source of Deployment are usually from management pack deployment which includes report deployment or building out the data warehouse (like creating outriggers, dimensions, fact tables, etc).  If you see an error in the event log, it usually has instructions on how to recover from it. For example, it may indicate you need to restart the services. In the Data Warehouse Management Server, there are three services just like on the Service Manager Management Group server:

·         System Center Data Access Service,

·         System Center Management,

·         System Center management Configuration

It’s usually best to recycle all of them at the same time.

Once your data warehouse is deployed, events are more likely to have a source of Data Warehouse. These events are written by jobs within the normal course of operations like the ETL jobs, the MPSync job and the DWMaintenance job. However, these all still write their events to the Operations Manager event log.

2. Data Warehouse Workspace in the Service Manager console

When you click on the Data Warehouse Jobs in the Data Warehouse workspace, you should see the ETL jobs and MPSync Job status. If your deployment was successful and your data warehouse is correctly registered to at least one Service Manager management group, you’ll see at least 5 jobs. Every job should either be in a status of “Running” or “Not Started”. 

If you see a job is in Failed status, you can click on this job and then click on Modules from the Tasks pane to find out which job module has failed. However to find out why it failed, the best place to get information from is the Operations Manager event log on the Data Warehouse server. You can also get more details from Powershell cmdlets which I will talk about in the next step. 

In the Data Warehouse workspace, you can also click on the Management Packs link in the left pane. Here’s where you can see all the management packs in the Data Warehouse and the status of their deployment. When you import a management pack to Service Manager, the MPSync job will synchronize it to the Data Warehouse (hence the name…MPSync means “management pack synchronization”). When you get the list of management packs in the data warehouse, you can find out if your management pack has been deployed successfully or not. 

If your management pack has defined data warehouse specific elements such as outriggers, dimensions, fact tables or reports, this Management Pack must be successfully deployed before the new tables and reports will be ready to use.

3. Powershell

The Powershell cmdlets provide much more detailed information about the Data Warehouse jobs than the console does.  This will likely change in a future release, but for now I really recommend you learn how to use the following useful cmdlets:

·         Get-SCDWMgmtGroup

This command tells us the sources which are registered with DW currently. We expect to see at least two different DataSourceName.

·         Get-SCDWJob

This command tells us all DW jobs status in the current batch.

In the above screenshot, you can check whether the jobs are enabled or not (see the highlight), which jobs are running and when they started. 

When MPSync Job or DWMaintenance job start, they both disable all of the ETL jobs so you will see the IsEnabled column set to false for each of the ETL jobs. This means that even if the ETL job status shows it is running, it actually isn’t. When MPSync Job or DWMaintenance job complete the ETL jobs will automatically get enabled and resume processing.

Jobs are normally in the “Not Started” status and if it is you can assume the previous batch has completed. If you’d like, you can use the following command to view the latest few batches of a specific job.

Get-SCDWJob –JobName <Specific job name>  -NumberOfBatches <number>

In the above screenshot, you can see the latest MPSyncJob completed, when it started and when it ended. If you’d like you can calculate how long it ran, and what the next batch id and status is.  The job batch id is always incremental.

·         Get-SCDWJobModule

This command provides detailed information about the specific modules within the job.  This is very useful when you see job failed and want to find out what caused the failure.

List of common issues and how to troubleshoot them

This list is not exhaustive, but it does cover most of the common issues customers have run into. This list will get added to over time so feel free to check back if you run into a new issue:

·         Reports aren’t deployed after registering the data warehouse

·         Job(s) fail after importing a custom management pack

·         Data warehouse isn’t getting new data or jobs seem to run forever

·         Custom data warehouse extensions don’t appear in the data warehouse

·         Management packs are stuck in “pending association” after registering the data warehouse

·         ETL jobs fail due to login credentials issues

This is the latest in “Troubleshooting the Data Warehouse” series. The goal of this series is to not only help you troubleshoot specific issues you may be experiencing but also teach you more about the data warehouse. For a complete list of posts in the series, check out Troubleshooting the Data Warehouse: An overview.

The symptom

After importing your management pack which defines some dimensions and/or fact tables to Service Manager, MPSync job has run several times but you still don’t see your dimension and/or fact tables in the DataMart.

The troubleshooting steps

To troubleshoot this issue, investigate the following things in order.

1. Ensure your management pack is sealed

MPSync Job can only bring sealed management pack from Service Manager to Data Warehouse. If you haven’t sealed your management pack, the simple solution is to seal it and then import it to Service Manager.

2. Ensure your management pack has synced to Data Warehouse

·         Open the Console

·         Click on the Data Warehouse wunderbar

·         Click on Management Packs then find your management pack in the list of management packs. To do this, use the search feature (type your management pack name in the search box).

If you don’t see your management pack,

·         It might have failed to import to Data Warehouse Management Server. Go to the Data Warehouse Management Server, open the Operations Manager event log and then filter the events with Event Sources as OpsMgr SDK Service.

·         The MPSync job may not have run yet. It runs on a recurring schedule, out of the box it runs every hour. You can modify the schedule with Powershell. In order to speed up the management pack synchronization, after import your management pack, you can manually resume MPSync job either from UI or powershell.

3. Check the deployment status of your management pack

If the management pack is sealed and has synced to Data Warehouse

·         Open the Console

·         Click on the Data Warehouse wunderbar

·         Click on Management Packs then find your management pack in the list of management packs. To do this, use the search feature (type your management pack name in the search box).

·         Check the deployment status (last column) of your management pack.

4. If the deployment status is Failed

Go to the Data Warehouse Management Server, open the Operations Manager event log and then filter the events with Event Sources as Deployment.

If there is error message, usually the error message will indicate what went wrong. If after correct the management pack, the error still occurs. In order not to affect the overall system, you can uninstall this management pack from Service manager Console, Administration > Management Packs. After MPSync job run, the management pack will be uninstalled from Data Warehouse Management Server.

This is the latest in “Troubleshooting the Data Warehouse” series. The goal of this series is to not only help you troubleshoot specific issues you may be experiencing but also teach you more about the data warehouse. For a complete list of posts in the series, check out Troubleshooting the Data Warehouse: An overview.

The symptom

I don’t see data or new data in my reports or ETL Jobs look like running forever and never in Not Started status.

The troubleshooting steps

To troubleshoot the above issue, please verify things in the following orders

1. Use the Get-SCDWJob PowerShell cmdlet to determine if all ETL jobs are enabled

Start PowerShell and type Get-SCDWJob

·         If the ETL jobs are disabled and either MPSyncJob or DWMaintenance job is running, you’ll need to wait a little bit to get the job status again (because these two background jobs disable the ETL jobs).  However, if these two jobs are Not Started and the ETL jobs are disabled, you can use the Enable-SCDWJob cmdlet to enable each of them. For example

Enable-SCDWJob -JobName Transform.Common

·         If ETL jobs, MPSync job and DWMaintenance job are all enabled and running, but their individual batch id has not changed for a long time or use Get-SCDWJobModule for specific job, we don’t see any module actually is running, check the event log and see if there is any error message. Sometimes, the error message might be many days old, we need to dig deep in the old day events.

·         Check if three services (System Center Data Access Service, System Center Management, System Center management Configuration) in Data warehouse Management server are actually running.

In Data Warehouse Management Server, click Start > Run .., enter Services.msc, in the Services window, verify if the following services are running or not

o   System Center Data Access Service

o   System Center Management

o   System Center management Configuration

If any of them are not running, restart all three services. In addition, if all services are actually running, events from Event Source Data Warehouse and OpsMgr SDK Service are outputted to Operations Manager event log. We can use this information as another source to verify if all services are running or not. If you don’t see events from Event Source Data Warehouse and OpsMgr SDK Service for a while, restart all three services regardless.

This is the latest in “Troubleshooting the Data Warehouse” series. The goal of this series is to not only help you troubleshoot specific issues you may be experiencing but also teach you more about the data warehouse. For a complete list of posts in the series, check out Troubleshooting the Data Warehouse: An overview.

The symptom

Some or all ETL jobs have failed. The Operations Manager event log in the Data Warehouse Management Server indicates the ETL job failure is related to a login user failure.

The troubleshooting steps

To troubleshoot this issue, investigate the following things in order.

1. Check if the password for each run as accounts has changed or expired.

If it is, you need to update it. To do this:

·         Click on Data Warehouse > Security > Run As Account, click on the related run as account, then click the Properties from the Tasks pane, update the Password field in the window and then click OK

·         If this run as account is Operational System Account, you also need to update the services that are running under this account.

o   In Data Warehouse Management Server, click Start > Run .., type Services.msc

o   In the Services window, update the passwords for the services that run under this account, for example,  System Center Data Access Service and System Center management Configuration

o   Restart the services

Note that MP Sync job and Extract job for Service Manager can use a different run as account other than Operational System Account. This run as account is created while Service Manager is registered to Data Warehouse. See the following Data Warehouse Registration Wizard, the New … button

It is easy to update the password if it is expired. However it is complicated to update the system if you change the run as account. We don’t recommend that you change run as accounts.

2. If the job failure is not related to password

You should make sure the run as account for the failed job can be used to connect to the target database. For example, ensure the Extract job run as account can be used to connect to the Service Manager database. If not, make sure the SQL service hosting the database is running.