Share via

Configure a dedicated front-end Web server for crawling by editing the Hosts file (Search Server 2008)

  • Article

Applies To: Microsoft Search Server 2008

 

Topic Last Modified: 2009-06-25

Before you perform this procedure, confirm that:

Use the following procedures in the order listed to configure a dedicated front-end Web server for crawling by editing the Hosts file directly.

In this article:

  • Ensure that the dedicated front-end Web server for crawling is not in the network load balancing rotation

  • Start the Windows SharePoint Services Web Application service on the front-end Web server

  • Specify the dedicated front-end Web server to use for crawling

  • Configure the Hosts file

Ensure that the dedicated front-end Web server for crawling is not in the network load balancing rotation

Because crawling content typically places the front-end Web server under heavy load, we recommend that you remove the dedicated front-end Web server from the network load balancing rotation. Otherwise, user requests that the network load balancer directs to the dedicated front-end Web server for crawling will be subjected to inconsistent performance. For more information about network load balancing, see Remove a host from a Network Load Balancing cluster (https://go.microsoft.com/fwlink/?LinkID=91030).

Start the Windows SharePoint Services Web Application service on the front-end Web server

You must ensure that the Windows SharePoint Services Web Application service is running on the front-end Web server that you want to use for crawling content.

Important

You must be a farm administrator to complete this procedure.

Starting the Windows SharePoint Services Web Application service

  1. In Central Administration, on the Operations tab, in the Topology and Services section, click Servers in farm.

  2. On the Servers in Farm page, in the Server column, click the name of the server that you want to configure as your dedicated front-end Web server for crawling.

  3. On the Services on Server page, in the Start services in the table below section, find the Windows SharePoint Services Web Application row. Verify that the Status column displays Started. If this service is started, no further action is needed. Otherwise, click Start.

Specify the dedicated front-end Web server to use for crawling

Important

You must be a server administrator to complete this procedure.

Before you can configure the Hosts file, you must collect the following information:

  • The base or host header name of each Web application in your farm that you will crawl.

  • IP address of the server that you want to use as the dedicated front-end Web server for crawling.

Obtain the base or host header names

Important

You must be a farm administrator to perform the following procedure.

Obtain Web application names

  1. In Central Administration, on the Application Management tab, in the SharePoint Web Application Management section, click Web application list.

  2. On the Web Application List page, in the URL column, note the base or host header names of the Web applications that you want to crawl. For example, if the URL is http://contoso, the base or host header name is contoso.

Determine the IP address of the dedicated front-end Web server for crawling

If you configure the index server as the dedicated front-end Web server for crawling, you can skip to Configure the Hosts file. Otherwise, note the IP address of the server that you want to configure as the dedicated front-end Web server for crawling. If you know the NetBIOS name of the server, you can use the ping command to determine the IP address.

Configure the Hosts file

Important

You must be a member of the administrators group on the index server to complete the following procedure.

Configure the Hosts file to specify a dedicated front-end Web server for crawling

  1. In Windows Explorer, navigate to the <systemdrive>:\winnt\system32\drivers\etc folder.

  2. Open the Hosts file in Notepad.

  3. On the line below the entry 127.0.0.1 localhost, add the entries for the Web applications that you want to crawl.

  4. On the File menu, click Save As.

  5. In the Save as type list, select All Files.

  6. Click Save and then close Notepad.

Hosts file examples

The following examples show how you might configure the Hosts file for different scenarios. These scenarios assume the following:

  • You have removed the server that you are specifying as the dedicated front-end Web server for crawling from network load balancing rotation, if applicable. For more information about network load balancing, see Remove a host from a Network Load Balancing cluster (https://go.microsoft.com/fwlink/?LinkID=91030).

  • You have started the Windows SharePoint Services Web Application service on the server that you are specifying as the dedicated front-end Web server for crawling, as described earlier in this article.

Scenario 1 - Configure the index server as the front-end Web server for crawling

Your server farm has three Web applications that you crawl. The application names are Contoso, Marketing, and Support. Because you are using the index server as the front-end Web server for crawling, you can use the loopback IP address (127.0.0.1) to specify the index server as the front-end Web server for crawling.

You add the following entries to your Hosts file below the entry 127.0.0.1 localhost:

127.0.0.1 Contoso

127.0.0.1 Marketing

127.0.0.1 Support

Scenario 2 - Configure a server other than the index server as the front-end Web server for crawling

Your server farm has four Web applications that you want to crawl. The application names are Contoso, Marketing, Sales, and Support. Because you are not using the index server as the Web front-end for crawling, you must add the actual IP address of the server that you want to specify as the dedicated Web front-end for crawling. In this example, the IP address of the dedicated front-end Web server for crawling is 10.10.10.4.

You add the following entries to your Hosts file below the entry 127.0.0.1 localhost:

10.10.10.4 Contoso

10.10.10.4 Marketing

10.10.10.4 Sales

10.10.10.4 Support

See Also

Concepts

Estimate performance and capacity requirements for Search Server 2008
Configure a dedicated front-end Web server for crawling (Search Server 2008)
Configure a dedicated front-end Web server for crawling by using the user interface (Search Server 2008)