SharePoint auto created managed properties and document processing performance

  • Article

Out of the box, SharePoint 2013 will automatically create a Managed Property when it discovers a Site Column in a SharePoint site. This TechNet article describes this feature: https://technet.microsoft.com/en-us/library/jj613136.aspx. This feature makes it easy for users who use a site column on multiple lists on their site to also search for that content based on the site column value.

In large search environments, this feature can create a large number of managed properties which will cause performance issues. The supported number of managed properties for a Search Service Application is 50,000. (see https://technet.microsoft.com/en-us/library/cc262787.aspx ). As the number of managed properties grows, the amount of time spent mapping crawled properties to managed properties during content processing also grows. This can have major impact on the rate at which new items can be added to the index.

What should I do to avoid this problem?

The performance impact of auto created managed properties has been mitigated in SharePoint 2016, so we recommend upgrading to SharePoint 2016 to avoid this performance issue.

If you cannot immediately upgrade to SharePoint 2016, you can disable the creation of managed properties in SharePoint 2013 to avoid the performance impact caused by a large number of managed properties. If you are already seeing a performance impact, then you would also need to delete some of your managed properties to reduce the performance impact during content processing. You'll find information on how to do that below.

Do I even have this problem?   The Content Processing Activity chart in the Crawl Health Reports in the Search Service Administration page can help identify how long the property mapping is taking. Here is an example of that chart showing extreme delays in content processing due to number of auto created managed properties which have been created

[caption id="attachment_3375" align="alignnone" width="640"]Content Processing Activity chart Content Processing Activity report on a SharePoint 2013 farm with 90,000 managed properties.[/caption]

To determine how many managed properties you have, browse to your Search Service Administration page in the Central Administration website and click Search Schema link. The total count is listed at the top of the list of Managed Properties.

You could also use the following PowerShell commands.

$mps = Get-SPEnterpriseSearchMetadataManagedProperty -SearchApplication "<SearchServiceApplicationName>"

$mps.count

I have too many properties, or I'm on the path to too many properties. How do I disable auto creation?

To disable the auto creation of managed properties for site columns, we will modify the settings on the “SharePoint” metadata category. The SharePoint metadata category is a collection of many property sets. Each property set can have its own settings, however the settings shown in Central Administration Website or PowerShell do not reflect the settings for each of these included property sets. The default is that the property set for site columns has this enabled, but the setting shown to the user reflects a different property set where the settings are disabled. The steps to disable this feature require that we enable auto creation at the category level, which will set all included property sets to enabled, then disable it on the category, disabling it on all property sets. This design means that once we have changed these settings we can NOT revert to the original behavior with individual property sets having unique settings. These changes can be made in Central Administration or via PowerShell.

Central Admin:

  1. From the Search Service Administration page, click Search Schema.
  2. On the Search Schema Page, Click Categories in the “breadcrumb” navigation at the top of the page.
  3. Locate the SharePoint category and use the dropdown menu to select Edit category. Only 50 categories are listed per page so you may need to navigate to additional pages to location the SharePoint category.
  4. Select the Searchable, Queryable, Retrievable checkboxes and click OK.
  5. Locate the SharePoint category and use the dropdown menu to select Edit category.
  6. Deselect the Searchable, Queryable, Retrievable checkboxes and click OK.

SharePoint PowerShell:

$mcat = Get-SPEnterpriseSearchMetadataCategory -SearchApplication "<SearchServiceApplicationName>" -Identity SharePoint

$mcat.AutoCreateNewManagedProperties = $true

$mcat.update()

$mcat.AutoCreateNewManagedProperties = $false

$mcat.update()

I've turned off auto creation, but how do I clean up this mess?  

If you find that the number of Managed Properties is too high and impacting your performance, you can remove some Managed Properties. You can remove them from either the Central Administration Website or PowerShell.

In Central Administration, locate the Managed Property in the Search Schema page and use the drop-down menu to delete the property.

In PowerShell, you cannot remove a Managed Property which still has mappings to a Crawled Property, so you must first remove those mappings. Here is an example of the steps a script would need to take.

  1. User Get-SPEnterpriseSearchMetadataManagedProperty to get the managed property you wish to remove.
  2. Retrieve the mappings for the Managed Property you wish to delete using Get-SPEnterpriseSearchMetadataMapping.
  3. Iterate through the mappings, removing each one using Remove-SPEnterpriseSearchMetadataMapping
  4. Once all mappings have been removed, remove the Managed Property with Remove-SPEnterpriseSearchMetadataManagedProperty

What else do I need to know?

One of the common questions I have heard is "How do I know which of my auto created Managed Properties are actually being used so I can only delete unused properties?" Unfortunately, I have not been able to discover a way to audit Managed Property use in queries. This means that there is some risk that you disrupt end user queries when you delete auto created Managed Properties. I would recommend that if you need to do so, you educate your helpdesk on how to recognize these queries and be ready to educate users on how to create site collection Managed Properties as well as a process in place where they can request Search Service Application level Managed Properties to be created for use cases that span multiple site collections.