How to configure a Dedicated Web Front End for crawling in SharePoint 2010

In SharePoint 2007 (MOSS v3) we have an option in the Office Server Search User Interface to configure a specific Web Front End as the server to be used for crawling. This functionality is very useful as it allows us to have a dedicated WFE for crawling as that ensures the crawling load is not distributed across the entire farm. Usually the indexer is configured with the Web Application services and is dedicated for crawling and then, this server is removed from the NLB .

Screen shot from SharePoint 2007

This option in SharePoint 2010 has been removed from the UI and is part of Powershell functionality. So how do you go about doing this? Here goes,

########## Start of Script to Configure a Dedicated Web Frontend in SharePoint 2010  ##########

$listOfUri = new-object System.Collections.Generic.List[System.Uri](1)

$zoneUrl = [Microsoft.SharePoint.Administration.SPUrlZone]'Default'

$webAppUrl = "Your Default Zone FQDN URL"

$webApp = Get-SPWebApplication -Identity $webAppUrl

$webApp.SiteDataServers.Remove($zoneUrl) ## By default this has no items to remove

$listOfUri.Add("Your Dedicated WFE Server URL");

$webApp.SiteDataServers.Add($zoneUrl, $listOfUri);

$WebApp.Update()

########## End of Script to Configure a Dedicated Web Frontend in SharePoint 2010  ##########

Now you might have a requirement to roll back this change, which means change the setting back to "Use all web front end computers for crawling". The script for doing that is as follows,

########## Start of Script to reset Dedicated Web Front end crawling settings to default in SharePoint 2010  ##########

$zoneUrl = [Microsoft.SharePoint.Administration.SPUrlZone]'Default'

$webAppUrl = " Your Default Zone FQDN URL "

$webApp = Get-SPWebApplication -Identity $webAppUrl

$webApp.SiteDataServers.Remove($zoneUrl);

$WebApp.Update()

########## End of Script to reset Dedicated Web Front end crawling settings to default in SharePoint 2010  ##########

Comments

• Anonymous
  January 03, 2011
  Regarding - $webAppUrl = "Your Default Zone FQDN URL" Should that be the load balanced URL? Regarding - $listOfUri.Add("Your Dedicated WFE Server URL"); what additional configuration do we need in AAM for this?

• Anonymous
  January 03, 2011
  $webAppUrl = "Your Default Zone FQDN URL" - This is your load balanced URL. $listOfUri.Add("Your Dedicated WFE Server URL");  - This is the URL of the WFE that you would like to dedicate for crawling.

• Anonymous
  April 28, 2012
  Hello Vijay, Let me explain my environment: There are 4 servers. 2 acting as WFE and 2 acting as Application servers (excel services, Web Analytics, search index). SPSAPPSRV1 - APP Server SPSAPPSRV2 - APP Server SPSWFESRV1 - WFE Server SPSWFESRV2 - WFE Server For my applications there are only one URL for load balance, as following: http://portal.domain.local http://intranet.domain.local http://blog.domain.local http://epm.domain.local/pwa Currently, when the crawling is started, it will be served for one of two WFE, then, the WFE that is serving for crawling the content, will rise the CPU usage to 100% (W3WP process) and if an user try to access one of these above URLs, will face a slow response. Because of this situation, I would like know in my scenario, how to add a third WFE and set it to be the dedicated WFE to crawling. Could you give me the sample in this script to perform the appropriated configuration in my environment? Thanks in advance!

• Anonymous
  June 27, 2012
  Hi Marco, What you are saying is that your crawler is currently hitting the load balanced URL and you would like to configure it for a single WFE (new one say WFE3). For this, you will first need to introduce the new WFE into the environment. I guess you know how to do this by running PSConfig and joining an existing farm. Once this happens, and the new WFE starts functioning do not add it to the load balancer. Now from the above script change it to include the new WFE name & FQDN name. ########## Start of Script to Configure a Dedicated Web Frontend in SharePoint 2010  ########## $listOfUri = new-object System.Collections.Generic.ListSystem.Uri $zoneUrl = [Microsoft.SharePoint.Administration.SPUrlZone]'Default' $webAppUrl = "http://portal.domain.local"    <- Change here $webApp = Get-SPWebApplication -Identity $webAppUrl $webApp.SiteDataServers.Remove($zoneUrl)          ## By default this has no items to remove $listOfUri.Add("http://WFE3");    <- Change here $webApp.SiteDataServers.Add($zoneUrl, $listOfUri); $WebApp.Update() ########## End of Script to Configure a Dedicated Web Frontend in SharePoint 2010  ##########

• Anonymous
  July 19, 2012
  Hi Vijay, Thanks for the info.  How would I modify the script if I would like to add more than one dedicated WFE for crawling?

• Anonymous
  July 22, 2012
  Hi Nate, This should take care of it, $listOfUri.Add("http://WFEFORCRAWL1");     $listOfUri.Add("http://WFEFORCRAWL2");     $listOfUri.Add("http://WFEFORCRAWL3");     Regards, Vijay

• Anonymous
  February 05, 2013
  The comment has been removed

• Anonymous
  February 10, 2013
  Hi Ashar2k11, I am guessing there is some confusion here. In SharePoint 2007, we had the following functionality in the UI, On the Configure Office SharePoint Server Search Service Settings on server page, in the Web Front End And Crawling section, we could select a computer under "Use a dedicated web front end computer for crawling". This would ensure that the crawler would target this WFE server only for crawling purpose. This option is not present in SharePoint 2010 and this article is to show how to do that in SharePoint 2010. Regards, Vijay

• Anonymous
  May 19, 2013
  The comment has been removed

• Anonymous
  May 20, 2013
  What I recollect is that once we set a dedicated crawler WFE, SharePoint itself adds a hosts entry wherein it will use the URL with the IP of the dedicated WFE in there. So I expect this to happen the minute you run the scripts above.

• Anonymous
  March 18, 2014
  The comment has been removed