Stage 4: Set up search and enable the crawling of your catalog content

  • Article

This is a blog post in the series “How to set up a product-centric website in SharePoint Server 2013”.  In this series, I'll use data from a fictitious company called "Contoso" to show you how to use search features to set up a website based on product catalog data.
Note: Most of the features described in this series are not available in SharePoint 2013 Online.

For an overview of the blog posts in this series, go to How to set up a product-centric website in SharePoint Server 2013.

Quick overview

As described in Stage 3: How to enable a list as a catalog, the Products list has now been enabled as a catalog. As we'll be using search technology to retrieve and display content on our publishing site (the Contoso website), we have to crawl the catalog so that its content is added to the search index.

In this blog post, we'll learn:

 

Start stage 4 

About crawling

Depending on your permission level, you can start a crawl from two places:

Where to start crawl Required premission level
Central Administration Search service application administrator
On the catalog Site collection administrator

The reason why you can start a crawl from two places is that people working with catalog content (let’s call them content managers) are not likely to have Search service application administration level rights, that is, they don’t have access to Central Administration.

When changes are made to catalog content, it makes sense to want to crawl this content fairly quickly, so that it can be displayed on the publishing site.

Luckily, content managers can make new content available without having to pester Search service application administrators to run new crawls -- but we'll be coming back to how to do this very shortly, as I first want to explain how to start a full crawl in Central Administration.

 

How to start a full crawl in Central Administration

Before you can start a full crawl in Central Administration, you have to specify which content source should be crawled. When you run a full crawl, all content in the content source is crawled even if that content has already been added to the search index.

For this scenario, we'll crawl the Local SharePoint sites content source.

  1. Go to Central Administration --> Manage service applications --> Search Service Application -- > Content Sources.
  2. On the Manage Content Sources page, hover over the Local SharePoint sites content source, and select Start Full Crawl from the menu.

Start full crawl

The status of the crawl is shown in the Status column.

  1. Refresh this page until you see that the value in the Status column is Idle.
    This means that the crawl has finished.

Crawl status

  1. Optionally, you can verify that your items have been added to the search index by clicking Crawl Log.
    In our scenario, we now have 870 items in the search index, which is approximately the same amount of products we have in the Products list.

Crawl log

 

 

 How to enable continuous crawls in Central Administration

You can only start a full crawl manually. Nobody wants the hassle of having to manually start a crawl every time a change is made to their catalog content, as this is neither an efficient nor practical way to work. So, to avoid this overhead, you can simply enable a continuous crawl of your content source that contains the catalog.

Continuous crawls start automatically at set intervals. Any changes that have been made to the catalog since the previous crawl, are picked up by the crawler and added to the search index.

To enable continuous crawls:

  1. Go to Central Administration --> Manage service applications --> Search Service Application --> Content Sources.
  2. On the Manage Content Sources page, click Your content source for which you want to enable continuous crawl, in our scenario case, this is Local SharePoint sites.
  3. Select the option Enable Continuous Crawls.

Enable continuous crawls

 

 

How to set continuous crawl interval

The default interval for continuous crawls is 15 minutes. You can set shorter intervals by using PowerShell. The code snippet below sets the continuous crawl interval to 1 minute.

$ssa = Get-SPEnterpriseSearchServiceApplication
$ssa.SetProperty("ContinuousCrawlInterval", 1)

So, by enabling continuous crawls, you can avoid a lot of frustration from content managers as they no longer have to wait for Search service application administrators to start a crawl for them. However, for some catalog changes, for example, enabling managed properties as refiners, continuous crawls are not sufficient, and you will need to do a full reindexing of the catalog content. But not to worry. Content managers have no reason for concern, because there is a way for them to initiate a full reindexing of the catalog.

 

 

How to initiate a reindexing of the catalog

To mark a catalog for reindexing, here's what to do:

  1. On your catalog (in our case the Products list in the Product Catalog Site Collection), click the LIST tab --> List Settings --> Advanced Settings.
  2. On the Advanced Settings page, click Reindex List.

Reindex list

 

 

How to view crawl status and schedule for a catalog

You can view the crawl status and schedule for an individual catalog. To do this:

  1. On your catalog (in our case the Products list in the Product Catalog Site Collection), click the LIST tab --> List Settings --> Catalog Settings.
  2. On the Catalog Settings page, you can see when the catalog was last crawled, and what crawls are scheduled to run when.
    In our case, we can see that the catalog was last crawled on 3/4/2013 at 5:30:17 AM, and that continuous crawls are scheduled to run every 15 minutes.

Crawl status for list

So, all in all, content managers can be happy because their content is added to the search index at short intervals, and Search service application administrators can be happy because they are no longer bothered by content managers constantly asking them to start a crawl.

 

 

Next blog article in this series From site column to managed property - What's up with that?

 

Additional resources