https://docs.microsoft.com/archive/blogs/scsmfromthefield/feed.xml Stuff that makes System Center Service Manager life easier Tue, 16 Apr 2019 06:16:26 GMT en-US hourly 1 https://docs.microsoft.com/archive/blogs/scsmfromthefield/scsm-2016-and-1801-authoring-tool Tue, 15 May 2018 08:36:20 GMT https://blogs.technet.microsoft.com/scsmfromthefield/?p=135 Visual Studio 2008 Shell Redist because the link in the prereq check is broken. The product group has recently re-released the authoring tool with these bits included so that customers do not have to go hunt for it on the internet. :) The downloads to the authoring tools are below:   2016: https://www.microsoft.com/download/details.aspx?id=54059 1801: https://www.microsoft.com/download/details.aspx?id=56561]]> https://docs.microsoft.com/archive/blogs/scsmfromthefield/scsm-roadmap-and-future Fri, 03 Nov 2017 06:46:58 GMT https://blogs.technet.microsoft.com/scsmfromthefield/?p=115 So, here's my response to anyone with this question: When System Center Configuration Manager was moved to a shorter semi-annual release cycle 2 years ago or so, it gave the PG the ability to develop features on a much shorter timeline so that the product could be compatible with new Windows client features. I think this made sense to most people. However, there was a lot of confusion about the remaining components of System Center and their supportability with Configuration Manager moving into this new release cadence on its own (e.g., which versions of Configuration Manager were compatible with each version of Service Manager).  So, a few months ago Microsoft announced that the remaining components of System Center (Including DPM, VMM, SCOM, SCSM and Orchestrator) would also be moving to a semi-annual release cycle, along with Windows Server. Here's the official release statement for 1801: https://azure.microsoft.com/en-us/blog/first-system-center-semi-annual-channel-release-now-available Why is this important? By releasing these products more frequently, the rest of System Center can now leverage the development agility that Configuration Manager has - meaning additional features and fixes released more frequently. On the flip side of that, this means the roadmap fundamentally changes as well. If features and fixes are being released semi-annually, it makes sense that the next set of features have about the same visibility. This means that the days of 3 year roadmaps for any System Center product are gone. What does this mean for you? System Center Service Manager and Orchestrator are still being developed and are part of this new release cycle along with the rest of System Center. Contrary to what some have said, they are definitely not currently in End of Life and currently have no forecasted EOL. Some semi-annual updates will only have fixes and some will have additional functionality. The features that get added to the entire suite each cycle will depend on customer demand and will be prioritized as such.  The products which receive enhancements will likely vary each time. All products are therefore still fully supported. Like many on-prem product groups, the SCSM team is currently working on incorporating more Azure and cloud service components into SCSM. Look at the service management connector for example: https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-itsmc-overview   Hope this helps. :)]]> https://docs.microsoft.com/archive/blogs/scsmfromthefield/pre-upgrade-tasks-for-scsm-2012-r2-to-2016 Fri, 09 Jun 2017 16:03:44 GMT https://blogs.technet.microsoft.com/scsmfromthefield/?p=105 https://docs.microsoft.com/system-center/scsm/upgrade-environment. However, before you upgrade to SCSM 2016, I highly recommend you check a few things so that you try to catch something that may cause the upgrade to fail. The first, is what is discussed here: https://blogs.technet.microsoft.com/servicemanager/2016/08/03/scsm-2016-upgrade-steps-for-custom-development/ This outlines what needs to be changed in custom solutions for them to work in SCSM 2016. I did this recently for a solution from the TechNet Gallery and it was pretty simple. Make sure that you plan for each of your custom add-ons to SCSM, as every free tool, portal and custom solution will need to be upgraded. Check with the developers of your tools (if not yourself) and see if they have a 2016-compatible version (most do). After verifying that you have the updated tools available, I usually run a few SQL Queries to see if anything in the ServiceManager database or DWStagingAndConfig is stuck. If there is a problem now, it will get worse with an upgrade (general rule of thumb). If you have a result from any of these, figure out why. Either post a comment, look it up online, or open a case. Run these queries against the ServiceManager database and DWStagingAndConfig database:   Select * from DeploySequenceView where DeploymentStatusId !=6 Select * from DeploySequenceStaging select * from DeployItemStaging Select * from infra.process p left join infra.Batch b on p.ProcessId = b.ProcessId left join infra.WorkItem w on w.BatchId = b.BatchId where w.StatusId NOT IN (3,6)   After checking the DB, check the OperationsManager event log on the workflow server. If you see errors like 33880 or 33333, figure out why they are happening. These and several other errors can indicate there is something corrupt in SCSM (like an SLO that hasn't been created through the console). Address these errors before upgrading. Lastly, MAKE SURE YOU HAVE YOUR RECOVERY KEYS!!!!! Hopefully this will help a few companies who would have otherwise had to implement a disaster recovery because the upgrade failed. -Chris Howie]]> https://docs.microsoft.com/archive/blogs/scsmfromthefield/scsm-reports-have-enum-values-for-lists Tue, 15 Nov 2016 10:33:15 GMT https://blogs.technet.microsoft.com/scsmfromthefield/?p=95 Antoni Hanus who helped take my raw process and turn it into a more friendly version. :) SCSM has an ETL (extract, transform and load) process which synchronizes the data from the operational database (ServiceManager) to the final data resting place (DWDataMart). Along the way, there are several tasks that the jobs take to help ensure they don't fail. One of these tasks is when the ETL moves list items through the DW, specifically within the transform job. List values are made up of their displaystring values and the GUIDs which represent them. What can happen is the GUID (enumeration id) can get to the transform job before the display string value. When this occurs, rather than failing, the transform job adds the enumeration id as both the id and the display string value. This helps, right? Well, not really, because we now have enum.GUID values in our reports. :( There are a few ways to address this issue. First, you could manually override the 'listname.value' column in the affected outrigger with whatever the value is supposed to be. This method works well if there are only a few affected values.  Unfortunately, there are typically many affected tables with many incorrect values. Versus trying to do all of these manually, this process below will resync all of them. BEFORE YOU START: This process can irreparably damage your data warehouse so be sure to backup all databases and follow this process exactly. This process is technically not supported unless a Microsoft engineer is on the phone. So, if you have an issue, you will need to restore the DBs and start over with a support case. PLEASE use this process at your own risk. I am publishing things like this because I know many people struggle with these issues. However, anything mentioned in this blog has no warranties, and is subject to the defined terms and conditions for TechNet blog content. With all of that said, you have been sufficiently warned :) Now, let's get to the good part! RESET ENUM PROCEDURE In summary, the high-level process is:

1. Backup databases
2. Disable the ETL job schedules so that the jobs do not try to run while we are performing this process
3. Delete the old extracted data from the inbound tables
4. Reset the job module watermarks for the ETL that are related to enumeration values
5. Run the ETL process
6. Verify all of the data is correct
7. Enable all of the schedules so that the jobs can resume

  STEP 1 Backup all of the Databases FIRST! Although changes will be made to only the DWStagingAndConfig database, it is wise to ensure all databases (ServiceManager, DWRepository, DWStagingAndConfig, DWDataMart, CMDWDataMart, OMDWDataMart are backed up to allow for any restore scenario / sync possibility that may need to be performed.   STEP 2 In PowerShell, disable the data warehouse job schedules so that they do not run until you want them to. On the DW Management Server, import the service manager warehouse cmdlets. Navigate to Service Manager installation directory e.g. Cd “C:\Program Files\Microsoft System Center 2012 R2\Service Manager” Once navigated to the appropriate directory, run the following command: import-module ".\Microsoft.EnterpriseManagement.Warehouse.Cmdlets.psd1" Run the following to get the schedules and set them to the $jslist variable: $jslist = Get-SCDWJobSchedule Run the following command to disable all job schedules: foreach($obj in $JSList) {Write-Host "Disabling Schedule for Job: " $obj.Name; Disable-SCDWJobSchedule -JobName $obj.Name;}   STEP 3 Verify all the jobs are in a status of ‘Not Started’ Run the following command and ensure all the job schedules are disabled: get-scdwjobschedule | format-table Also, be sure that none of them are running before starting. (This is huge! If you do this process while the jobs are running the DW jobs could get irreparably stuck) get-scdwjob | format-table NOTE: 3 or ‘Not Started’ is what you should see for all of the jobs prior to continuing. Once you have disabled the schedules, the jobs will not rerun after they complete their current execution. For documentation's sake, below are all of the job and module statuses that are used in the DW and their numerical value. This is especially beneficial when you are looking in the infra.process, infra.batch, and infra.workitem tables in the DWStagingandConfig DB.  

1	Success
2	Failed
3	Not Started
4	Running
5	Stopped
6	Completed
7	Waiting

  STEP 4 Query Number of Display Strings in Service Manager DB Check and note the number of rows in the display string view in the SCSM DB, by using the following command. Use ServiceManager select count(DisplayName) from ServiceManager.dbo.DisplayStringView This should output the total number of display strings in the Service Manager database (should be around 210k). This will be the number we will check for after each execution of the extract job in Step 7.   STEP 5: In the DWStagingAndConfig DB, Clear the inbound tables for enum values and validate they are empty: To prepare for the data reset, we need to clear everything related to enumerations that was previously extracted so that we can do a full resync. Run this against the DWStagingandConfig DB: Use DWStagingAndConfig delete from inbound.EnumType delete from inbound.DisplayString Verify they are now empty with these commands: select * from inbound.EnumType select * from inbound.DisplayString   STEP 6 : In the DWStagingAndConfig DB, Reset the Extract watermarks for the enums and display strings so that all of them can be resynced from SCSM: The first step in the ETL process is Extract. So, we start by resetting the extract job module watermarks which are related to enumeration values: Use DWStagingAndConfig update etl.WarehouseModuleDependency set WaterMark = '2009-01-31 00:38:11.753' where WarehouseEntityId IN (select WarehouseEntityId from DWStagingandConfig.etl.WarehouseEntity where  (WarehouseEntityName like '%enumtype%' or WarehouseEntityName like '%displaystring%') and SourceId = 3 and WarehouseEntityTypeId = 6) IMPORTANT: It is fine to use the date shown above for the query.   STEP 7: Run the Extract Job manually via PowerShell commands Back in PowerShell on the Data Warehouse Management Server, Run the Main Management Group’s Extract Job (This will be Extract_ManagementGroupName not the Extract_DW_ManagementGroupName) NOTE: In the command below change Extract_ManagementGroupName to the name of your extract job (run get-scdwjob if needed to determine the job name). Start-SCDWJOB -JobName Extract_ManagementGroupName Then run a get-scdwjob to ensure it shows as status = Running (or 4): get-scdwjob | format-table Re-run the get-scdwjob every 5 / 10 minutes or so until it changes to ‘Not Started’.   STEP 8: Verify Count of rows in Inbound.displayString table Run this Query in the DWstagingandConfig database: Use DwStagingAndConfig select count(DisplayName) from DWStagingAndConfig.inbound.DisplayString This typically is around 50,000 / 52.000 after 1 extract job (this is predictable, due to the number of records we process per batch.)   STEP 9: Repeat Steps 7 and 8 until the result in Step 8 matches that which you noted in STEP 4 Repeat Steps 7 and 8 running the extract, verifying the number in SQL until the number in SQL matches the number that was recorded in Step 4. This will typically take a total of 5 runs of the extract job to get to that point.   STEP 10 Reset the Transform and Load watermarks for all outriggers Now that the data is in the DWStagingandConfig DB, it is time to transform and load it into the DWRepository and DWDataMart. Run the following SQL statements 1 by one to reset the Transform and load watermarks for all outriggers: Use DWStagingAndConfig update etl.WarehouseModuleDependency set WMBatchId  = 0 where ModuleId IN (select ModuleId from etl.WarehouseEntity Entity inner join etl.WarehouseModule module on module.TargetWarehouseEntityId= Entity.WarehouseEntityId where  Entity.WarehouseEntityTypeId = 3)   Use DWStagingAndConfig update etl.WarehouseModuleDependency set CompletedBatchId  = 0 where ModuleId IN (select ModuleId from etl.WarehouseEntity Entity inner join etl.WarehouseModule module on module.TargetWarehouseEntityId= Entity.WarehouseEntityId where  Entity.WarehouseEntityTypeId = 3)   Use DWStagingAndConfig update etl.WarehouseModuleDependency set WaterMark = '2009-01-31 00:38:11.753' where ModuleId IN (select ModuleId from etl.WarehouseEntity Entity inner join etl.WarehouseModule module on module.TargetWarehouseEntityId= Entity.WarehouseEntityId where  Entity.WarehouseEntityTypeId = 3)   STEP 11 : Manually Execute Transform.Common Job Back in PowerShell on the Data Warehouse Management Server, Run the Transform.Common Job: Start-SCDWJOB -JobName Transform.common Then run a get-scdwjob to ensure it shows as status = Running (or 4) get-scdwjob | format-table Re-run the get-scdwjob  command every 5  minutes or so until it changes to ‘Not Started’.   STEP 12: Verify the enum values are no longer in the DWRepository The schema in the DWRepository is the same as the DWDataMart. So - after the transform job has run, you should be able to check one of the affected outriggers and verify that the enum values are no longer present. I like to do this before running the load job because there isn't a point in doing that if the values aren't fixed in the DWRepository.   STEP 13: Manually Execute Load.Common Job Back in PowerShell on the Data Warehouse Management Server, Run the Load.Common Job: Start-SCDWJOB -JobName Load.common Then run a get-scdwjob to ensure it shows as status = Running (or 4) get-scdwjob | format-table Re-run the get-scdwjob every 5 / 10 minutes or so until it changes to ‘Not Started’.   STEP 14: Verify outriggers in the DWDataMart Check the outriggers in the DWDataMart and ensure their values no longer contain enumeration ids. For any enums that exist in the DWDataMart but no longer exist in the ServiceManager DB, these will remain until manually cleaned out of the DWDataMart Database. If they are still used on workitems, you will need to do a manual update on the value in the outrigger or write them out of your reports using a NOT LIKE or something similar in the WHERE portion of your queries.   STEP 15: Re-enable job schedule for MPSyncJob It is important to turn the MPSyncJob and DWMaintenance jobs back on by themselves. If they try to run together, the DW jobs can get stuck. Back on the Data Warehouse Management Server, Enable the MPSyncJob schedule and run it: Enable-SCDWJobSchedule -JobName MPSyncJob Start-SCDWJob -JobName MPSyncJob Then run a get-scdwjob to ensure it shows as status = Running (or 4) get-scdwjob | format-table Re-run the get-scdwjob every 5 / 10 minutes or so until it changes to ‘Not Started’.   STEP 16: Re-enable job schedule for DWMaintenance Back on the Data Warehouse Management Server, Enable the DWMaintenance schedule and run it: Enable-SCDWJobSchedule -JobName DWMaintenance Start-SCDWJob -JobName DWMaintenance Then run a get-scdwjob to ensure it shows as status = Running (or 4) get-scdwjob | format-table Re-run the get-scdwjob every 5 / 10 minutes or so until it changes to ‘Not Started’.   STEP 17: Re-enable the remaining Job Schedules Run the following to get the schedules and set them to the $jslist variable: $jslist = Get-SCDWJobSchedule foreach($obj in $JSList) {Write-Host "Enabling Schedule for Job: " $obj.Name; Enable-SCDWJobSchedule -JobName $obj.Name;}   STEP 17 Create Test Incident (or work item) to monitor and ensure syncing is back to normal. Create a test incident so you can ensure data is syncing across normally as expected. Data should be synced all the way through from the live ServiceManager DB to DWDataMart in no more than 2 hours. You can also run the ETL manually to push things along. The following query can be run to easily observe the most recent incidents: Use DWDataMart select * from incidentdimvw order by createddate desc   YOU'RE FINISHED! :) I hope this helps some folks that are struggling with this.        ]]> https://docs.microsoft.com/archive/blogs/scsmfromthefield/scsm-html-5-ssp-cache-settings Thu, 27 Oct 2016 10:44:22 GMT https://blogs.technet.microsoft.com/scsmfromthefield/?p=65 UserCacheTimeout   The portal uses a caching infrastructure to provide a swift user experience. The value of this key defines the timeout, in seconds, to cache user-specific details of the signed-in user.   DataCacheTimeout   The portal uses a caching infrastructure to provide a swift user experience. The value of this key defines the timeout, in seconds, to cache generic data which can be shared among different users.     For information about all the other configurable portal settings, see https://technet.microsoft.com/en-us/library/mt622142(v=sc.12).aspx Hope this helps!]]> https://docs.microsoft.com/archive/blogs/scsmfromthefield/get-the-most-recent-assigned-to-user-or-any-other-relationship-with-one-to-many-cardinality Wed, 26 Oct 2016 16:27:14 GMT https://blogs.technet.microsoft.com/scsmfromthefield/?p=25 Hello, everyone! My name is Chris Howie, Microsoft PFE and current Technology Manager for SCSM in the United States. For nearly the past two years, I have funneled my bloggable content through other PFEs (primarily Brian Barrington and Antoni Hanus), mostly because they already had established blogs and I of course did not. Now, we are all so busy that I am going to have to carry my own weight in this regard. :) In summary, I have been putting off blogging for too long and am ready to get going on my own. Anyone that has had an onsite or remote engagement with me knows my passion around reporting in SCSM. Unfortunately, there are few out-of-the-box reports and they of course do not accommodate customizations, which most customers do, which results in me doing a TON of custom reporting. For my first post, I am going to address an issue which has plagued many customers who are attempting to retrieve the most recent instance of a relationship which is one-to-many. First, we must level-set on some terminology and concepts. Everything in SCSM is an object of a class (i.e. an Incident, Service Request, Windows Computer, etc.). These classes are related to each other via relationships. In the case of an Incident being related to a User, there is an Incident object and a User object and they are related to each other in a specific way through a relationship identifier (i.e. Assigned To User, Created By User, Affected User, Resolved By User). Some relationships can be many-to-many, such as About Configuration Item, which is the affected configuration items on work items, or one-to-many such as Assigned To User. Note: If you ever wanted to know the cardinality for a specific relationship, you can always look it up by running SELECT * FROM RelationshipType against the ServiceManager Database. There you will find the minimum and maximum cardinality settings for every relationship in SCSM. With many-to-many relationships, there can be multiple objects related to multiple others. For example, you can have multiple configuration items on an Incident, and those configuration items can also be related to multiple incidents. Alternatively, with one-to-many relationships such as Assigned To User, there can only be one active Assigned To User on an Incident, but that Assigned To User can be assigned to multiple Incidents. To maintain this single cardinality on work items, the existing relationship must be destroyed/deleted to allow the new one to exist. Keep this concept in your mind for the next section. :) So - why did I just tell you all of this? It is important to understand the two different relationship concepts in reporting because it ultimately determines how you need to consume the data in the data mart. In the DWDataMart, we keep relationship changes in the fact tables which hold them. For example, in the case of WorkitemAssignedToUserFactvw, if a work item was reassigned multiple times then there are multiple entries. This can be a good or bad thing, depending on the type of report you are wanting to write. For example, if you are wanting to include all of the people who were involved in a work item's life, then you would simply join to include all of them and move on. However, if you want the last assigned to user, or the last of any other one-to-many relationship, you have to do some determination. When viewing the WorkItemAssignedToUserFactvw and other fact views, you will likely notice that there is a DeletedDate column. This column is populated with the date/time of when the destroy process occurs for an existing one-to-many relationship that we previously discussed. So - the active (last) relationship should have a value of NULL when it has not yet been deleted, right? Well - yes and no. When the work item still exists in SCSM, the active relationship still exists also, therefore having a DeletedDate of NULL. However, when the work item is groomed, that relationship is also deleted, and therefore has a DeletedDate equal to the date/time it was groomed. Unfortunately, the OOB reports in 2012R2 and most customers' reports expect the DeletedDate = NULL to identify the last relationship. This means that as work items are groomed, the reports will no longer show the Assigned To User because the DeletedDate is now populated. Now what? Now that you understand the fundamental problem, you understand that we cannot simply use DeletedDate = NULL nor can we simply Max() the DeletedDate column because we would not include the NULL values on the active work items. To achieve the best of both worlds, I have come up with an interesting way to get the most recent assigned to user in both scenarios within a single query. First, I use the SQL functions Coalesce(which replaces NULL with a value) and GETDATE(which gets the current date/time) to convert the NULL DeletedDate values to the current date/time (which would make them the most recent). Then, I use the Max function to get the most recent date/time for each work item. This ensures that I have the last Assigned To User, whether it is active and NULL or it has been groomed. Below is an example of using this concept.   --This query retrieves the last Assigned To User on all Work Items in the DWDataMart SELECT WI.Id AS [WorkItem ID], AssignedInfo.DisplayName AS [Assigned To User] FROM WorkItemDimvw WI LEFT JOIN (Select Max(COALESCE(DeletedDate,getdate())) as [max deleted date],WorkItemDimKey from WorkItemAssignedToUserFactvw assignedFact group by WorkItemDimKey) assignedfact on wi.WorkItemDimKey = assignedfact.WorkItemDimKey LEFT JOIN WorkItemAssignedToUserFactvw assignedfact2 on WI.WorkItemDimKey = assignedfact2.WorkItemDimKey AND Coalesce(assignedfact2.DeletedDate,GETDATE()) = assignedfact.[max deleted date] LEFT JOIN UserDimvw AssignedInfo on AssignedFact2.WorkItemAssignedToUser_UserDimKey = AssignedInfo.UserDimKey where WI.Id IS NOT NULL   Thanks for reading, I hope this makes sense and helps those who are writing custom reports for SCSM! Have any ideas for me or the other PFEs to blog about? If so, please put them in the comments and we will do our best to get the knowledge out there!! :) -Chris Howie     This is provided "AS IS" with no warranties, and confers no rights. Use of included script samples are subject to the terms specified in the Terms of Use.]]>