Share via

Manage deletion of index items (SharePoint Server 2010)

  • Article

 

Applies to: SharePoint Server 2010

Summary: Configure four deletion policies to specify how the system deletes content from the index when the crawler encounters errors.

Microsoft SharePoint Server 2010 prevents accidental deletion of content from the index when the crawler encounters errors during a full crawl or an incremental crawl. The system uses four deletion policies to control accidental deletion of content. You can manage the four deletion policies at the service application level to specify how many times and how long an item will be retried in consecutive crawls.

These deletion policies existed in Microsoft Office SharePoint Server 2007, but you could not view them nor change their values.

We recommend that you use the default values listed in the following table as a starting point for a typical SharePoint Server 2010 installation. You might consider specifying values other than the default values in the following circumstances:

  • When scheduled maintenance prevents a network asset from being crawled.

  • When server or network policies extend the time it takes to complete a crawl.

  • When network latency causes crawls to exceed the value that is specified for a deletion policy.

  • When you know about the state of a previously crawled item and want to include the item in search results regardless of its current availability to the crawler.

The following table shows the policy names, strings, and default values.

Deletion policy name String Default value

Delete policy for access denied or file not found

ErrorDeleteCountAllowed

ErrorDeleteIntervalAllowed

30

720 Hours (30 days)

Delete policy for all other errors

ErrorCountAllowed

ErrorIntervalAllowed

100

1440 Hours (60 days)

Delete unvisited policy

DeleteUnvisitedMethod

1

Re-crawl policy for SharePoint content

RecrawlErrorCount

RecrawlErrorInterval

10

360 Hours (15 days)

The following table describes what the system does for each policy.

Deletion policy name Description

Delete policy for access denied or file not found

When the crawler encounters an access denied or a file not found error, the item is deleted from the index if the error was encountered in more than ErrorDeleteCountAllowed consecutive crawls AND the duration since the first error is greater than ErrorDeleteIntervalAllowed hours. If both conditions are not met, the item is retried.

Delete policy for all other errors

When the crawler encounters errors of types other than access denied or file not found, the item is deleted from the index if the error was encountered in more than ErrorCountAllowed consecutive crawls AND the duration since the first error is greater than ErrorIntervalAllowed hours.

Delete unvisited policy

During a full crawl, the crawler executes a delete unvisited operation in which it deletes items that are in the crawl history that are not found in the current full crawl. You can use the DeleteUnvisitedMethod property to specify what items get deleted. You can specify the following three values:

  • 0, all unvisited items are deleted.

  • 1 (default), unvisited items that have the same host as the start address specified in the content source are retained, and unvisited items that were discovered by following links to other hosts are deleted.

  • 2, none of the unvisited items are deleted.

Re-crawl policy for SharePoint content

This policy applies only to SharePoint content. If the crawler encounters errors when fetching changes from the SharePoint content database for RecrawlErrorCount consecutive crawls AND the duration since first error is RecrawlErrorInterval hours, the system re-crawls that content database.

To view and change deletion policies by using Windows PowerShell

  1. Verify that you meet the following minimum requirements: See Add-SPShellAdmin.

  2. On the Start menu, click All Programs.

  3. Click Microsoft SharePoint 2010 Products.

  4. Click SharePoint 2010 Management Shell.

  5. At the Windows PowerShell command prompt, type the following commands:

    $SearchApplication = Get-SPEnterpriseSearchServiceApplication -Identity "<SearchServiceApplicationName>"

$SearchApplication.GetProperty("<PropertyName>")

$SearchApplication.SetProperty("<PropertyName>", <NewValue>)

  6. Each of these property values can be changed for an individual Search service application by using GetProperty and SetProperty cmdlets. Repeat this procedure for each Search service application for which you want to change deletion policies.

For example, as a search administrator, you know that one of your remote Web servers is not available due to hardware failure and repair is scheduled to occur in six weeks. This down time will result in many search results being removed from the search index. Once the Web server is back online, restoration of the removed search results will require a full crawl which takes three days to complete. You want to avoid search results being dropped from the index, and you want to prevent the system from devoting resources to a full crawl. Therefore, you use the following procedure.

Deletion policy example

  1. Verify that you meet the following minimum requirements: See Add-SPShellAdmin.

  2. Get the search service application: $SearchApplication = Get-SPEnterpriseSearchServiceApplication -Identity "<SearchServiceApplicationName>"

  3. Get the default value of the property (Default is 720 hours): $SearchApplication.GetProperty("ErrorDeleteIntervalAllowed")

  4. Set the value of the property to 6 weeks expressed as 1008 hours: $SearchApplication.SetProperty("ErrorDeleteIntervalAllowed", 1008)

  5. Verify the new value of the property (New value is 1008 hours): $SearchApplication.GetProperty("ErrorDeleteIntervalAllowed")

    This new property value prevents search results from being removed from the search index and avoids the need for a full crawl.

See Also

Reference

Get-SPEnterpriseSearchServiceApplication
Get-SPEnterpriseSearchAdministrationComponent
Set-SPEnterpriseSearchAdministrationComponent
Ping-SPEnterpriseSearchContentService