SP2013 - Boost your FileShare incremental crawls

Hi Search Enthusiasts

I wanted to touch base on a particular hidden feature our SP2013 crawl is leveraging to improve File Shares incremental crawl performance.

Before going into details, as a pre-requisite, you will need to confirm that a parent folder modified date changes if one document within that folder is modified.

Some old file systems weren't doing so thus our crawl was forced to go through all folders contents every time. The consequence of that behavior is that we were crawling folders, impacting incremental crawls and send unnecessary operations to the Search index.

SP2013 July CU 2014 introduced a new registry key to allow the crawl to not index non modified folders during incremental crawl. As such the performance of an incremental crawl is much improved.

 HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\15.0\Search\Global\Gathering Manager

UseLastModifiedForFolders        dword:00000000

Restart your OSearch15 service for the change to be picked up on all crawl servers.

This new registry key indicates whether or not the incremental crawl can be based on the modified date time of the folder.

• By default (0), non modified folders are crawled during inc. crawl regardless of their modified datetime (legacy file shares mostly).
• Setting this to 1 would only crawl folders with a modified datetime superior to the last inc. crawl date. This provides stronger inc. crawl performance.

Happy Crawls.

Keep in Search

Comments

• Anonymous
  June 16, 2017
  Hi,impossible to find this registry key. Where is it supposed to be ?Didn't find any information elsewhere on this. Googled "UseLastModifiedForFolders" : 1 result (the current page). Weird...Could you help please ?
  • Anonymous
    July 25, 2017
    Hi Teng, Sorry for that, it seems the Key path didn't make it to the public post. The Key path is [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\15.0\Search\Global\Gathering Manager]As this is a new key, you have to create it. let me know