Using Enterprise Search Property Filters in SharePoint Server 2007

  • Article

Summary: Learn to use Enterprise Search property filters to narrow the focus of keyword searches in Microsoft Office SharePoint Server 2007. (23 printed pages)

Joel Krist, Akona Consulting

July 2008

Applies to: Microsoft Office SharePoint Server 2007

Contents

  • Using Keyword Query Syntax and Property Filters with Enterprise Search

  • Setting the Context

  • Running the Code Example

  • Conclusion

  • Additional Resources

Using Keyword Query Syntax and Property Filters with Enterprise Search

Enterprise Search in Microsoft Office SharePoint Server 2007 supports the creation of custom search applications. You can access search functionality programmatically in two ways:

Enterprise Search allows you to build queries by using three different types of query syntax:

  • Keyword syntax, in which you pass search terms directly to the search service.

  • SQL syntax, which you can use to construct complex search queries.

  • URL syntax, which allows you to pass queries directly to the search page by encoding the query in the URL.

This article examines how to use keyword query syntax with property filters to narrow the focus of keyword queries. The article provides example code that shows you how to filter searches by using managed properties, search scopes, and duplicates filtering.

To give you the context you need to understand how to best use property filters with keyword queries, this article provides the following information:

  • A walkthrough of the process of creating a document library with an associated search scope and managed property.

  • Step-by-step procedures to create a duplicate document condition by using a Microsoft Office Word document that you upload twice with different file names to the same document library.

  • Example code that performs keyword queries by using property filters to narrow the focus of searches and to handle duplicate search results.

Setting the Context

Before you can use the example code, you must create the document library, search scope, and managed property that the code will use.

The following sections walk you through the process of creating these items. To jump ahead to the query-building code, see Running the Code Example.

Creating the Document Library

The document library you create includes a custom column. Later, you add a managed property that maps to this custom column.

To create the document library

  1. Open the SharePoint site in the browser. On the Site Actions menu, click View All Site Content.

  2. Click Create on the All Site Content page.

  3. Click the Document Library link on the Create page.

  4. On the New page, name the new document library Word Documents, set the Document Template to Microsoft Office Word Document, and then click Create.

    Figure 1. Create new document library

    Create new document library

  5. In the All Documents view of the new document library, click Settings on the Document Settings menu.

  6. Click the Create column link in the Columns category, create a Single line of text column, and then name it Column1.

Now, you add two documents to the document library. To do this you create two simple Office Word 2007 documents that each contain a unique word that you can query for.

To create the Word documents

  1. Start Office Word 2007.

  2. Use the =lorem(10) function to enter 10 paragraphs of lorem ipsum text into the document. To do this, type the characters =lorem(10) into the document and press ENTER.

    Figure 2. Word document content

    Word document content

  3. We used the =lorem() function to add multiple words to the document instead of a single unique word because the algorithm used by the search service to identify duplicate documents is sensitive to word count. To explore working with duplicate search results, the search service must be able to recognize your two documents as the same document. In very short documents (for example, a document that contains a single word), search does not always recognize the documents as duplicates. Including the additional content enables search to make this distinction.

  4. Save the Word document to a folder on the local computer, and name it Test.docx.

  5. Save the Word document again, and name the document Test1.docx.

  6. Upload both documents to the Word Documents library you created in step 4. You can upload the documents from within Word by using Save As, or through the document library user interface in SharePoint Server. The end result is two instances of the same document that are located in the document library.

  7. Set the Column1 property of Test.docx to the string "Column 1 for Test".

  8. Set the Column1 property of Test1.docx to the string "Column 1 for Test1". Figure 3 shows properties of the documents in the document library.

    Figure 3. Document with properties set

    Document with properties set

Creating the Search Scope

After you create the document library and add two documents to it, you create a search scope that is associated with the library. Search scopes allow you to limit searches to a subset of the indexed content. Scopes can range in complexity from those created around a specific content source to those that are based on rules by using custom metadata. The keyword query syntax supports the Scope filter to so that you can limit a query to a specific scope.

To create the search scope

  1. Open the SharePoint Central Administration site in the browser, and then select the Shared Services Provider (SSP) to perform queries against. For this procedure, use the default SSP named SharedServices1.

    Figure 4. Selecting the SSP

    Selecting the SSP

  2. On the Shared Services Administration page for the SSP, click the Search settings link.

  3. On the Configure Search Settings page, click the View scopes link in the Scopes category.

  4. On the View Scopes page, click the New Scope link.

  5. On the Create Scope page, name the new scope Word Documents, and then click OK to create the scope and return to the View Scopes page.

    The new scope does not yet have rules associated with it. Click the Add rules link to open the Add Scope Rule page.

    Figure 5. Add rules link

    Add rules link

  6. Click Web Address for Scope Rule Type. Click the Folder option for the Web Address setting, and then type the URL of the document library that you created earlier.

    Figure 6. Add Scope Rule page

    Add Scope Rule page

  7. Click OK to save the rule and return to the View Scopes page. The scope rule is added, and the scope will be ready to use after the next update. To update now, return to the Search Settings page, and click Start update now.

    Figure 7. Start update now link

    Start update now link

Creating the Managed Property

You create a managed property so that you can filter your searches by values in the custom column that you added to your document library. Typically, you use a managed property to map multiple properties that contain the same information to a single property. This makes it easier to work with multiple properties and makes crawled properties available for searching. To explore search filtering by using managed properties, you can simply create a managed property that maps to only the custom column in your document library.

To create the managed property

  1. Start a search crawl so that the document library's custom column is available when you are creating the managed property column mapping. To perform the crawl, open the SharePoint Central Administration site in the browser, and then select the SSP to perform queries against.

  2. On the Shared Services Administration page for the SSP, click the Search settings link.

  3. On the Configure Search Settings page, click the Content sources and crawl schedules link in the Crawl Settings category.

  4. On the Manage Content Sources page, click Start Full Crawl for the Local Office SharePoint Server sites content source.

    The time it takes the crawl to complete depends on the amount of content to be crawled; however, the crawl can take a while.

    Figure 8. Start Full Crawl

    Start Full Crawl

  5. After the crawl is complete, navigate to the Configure Search Settings page and click the Metadata property mappings link in the Crawl Settings category.

  6. On the Metadata Property Mappings page, click the New Managed Property link.

  7. Name the managed property WordDocumentsColumn1 and leave its type as Text.

    Figure 9. Managed property name and type

    Managed property name and type

  8. Click Add Mapping in the Mappings to crawled properties category.

  9. In the Crawled property selection dialog box, type Column1 for the crawled property name, and then click Find. The ows_Column1(Text) property is displayed. This is the internal name of your custom column.

    Figure 10. Crawled property selection dialog box

    Crawled property selection dialog box

  10. Select the ows_Column1(Text) property, and then click OK to close the dialog box. The selected property is displayed in the list of mapped properties on the New Managed Property page.

    Figure 11. Crawled property mapping

    Crawled property mapping

  11. Click OK to save the managed property.

  12. Perform another full crawl so that you can do searches filtered by the managed property (see steps 1 through 4).

Now, all of the pieces you need are ready. Let's take a look at the code.

Running the Code Example

The code example and its instructions are based on the assumption that you are using Microsoft Visual Studio 2008. Some of the screens might look slightly different if you are using an earlier version of Visual Studio, but the basic ideas and processes are the same.

The following section provides details on the references and namespace using statements you need to add to cut and paste the example code into your projects.

Adding References and Namespaces

To show how to perform search queries via the search object model, the following code uses classes from the Microsoft.Office.Server.Search.Query namespace. Because the classes used are defined in the Microsoft.Office.Server.Search.dll assembly, you must add a reference to it and to the Microsoft.Office.Server.dll assembly that it depends on.

Table 1 lists the references required by the code and provides the path to the assemblies in a default SharePoint installation.

Table 1. Required references

Assembly Default Location

Microsoft.Office.Server.dll

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\ISAPI

Microsoft.Office.Server.Search.dll

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\ISAPI

Microsoft.SharePoint.dll

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\ISAPI

Note

Microsoft Office SharePoint Server 2007 includes the Enterprise Search assemblies listed in Table 1, and also includes the assemblies for search in Windows SharePoint Services. The classes for search in Windows SharePoint Services are implemented in the Microsoft.SharePoint.Search.dll assembly. When running on SharePoint Server, ensure that you are using the Enterprise Search assemblies in SharePoint Server. If you inadvertently add references to the search assemblies in Windows SharePoint Services, changes you make to content will not be reflected in your search results when you query via the object model, even if you perform a crawl after making the content changes. When you add references to the search assemblies in Windows SharePoint Services, and then kick off a crawl from the SharePoint Server user interface, your query will return results from the Windows SharePoint Services search index instead of results from the SharePoint Server search index. If you add the correct references to the search assemblies in SharePoint Server, everything should work as expected.

The example code also shows how to perform search queries via the Enterprise Search Query Web service (see Enterprise Search Query Web Service Overview). To make calls to the Query Web service, you need to add a Web reference in Visual Studio that creates a proxy class you can use in your code to make calls to the Web service methods.

The following steps show how to add a Web reference to the Enterprise Search Query Web service in Visual Studio 2008.

To add a Web reference to the Enterprise Search Query Web service

  1. In Visual Studio 2008, click Add Web Reference on the Project menu.

  2. In the Add Web Reference dialog box, type the URL of the service. The service is located at the following URL: http://Server_Name/[sites/][Site_Name/]_vti_bin/search.asmx. Replace Server Name and the optional Site Name with values that make sense for your environment.

  3. Click the Go button to make Visual Studio retrieve and display the service's description.

    Figure 12. Add Web Reference dialog box

    Add Web Reference dialog box

  4. Name the Web reference. The following example code uses SearchService for the name.

  5. Click Add Reference to generate the proxy class and add it to your project.

In addition to the references listed earlier, the example code requires the following namespace using statements to compile successfully.

using System.Net;
using System.Text;
using Microsoft.SharePoint;
using Microsoft.Office.Server.Search.Query;

Searching with the Query Web Service

To perform keyword searches by using the Query Web service, you need to create an instance of the proxy class that was generated when you added the Web reference to your project.

The following code shows how to instantiate the proxy class and set the credentials that are used when calling the service.

SearchService.QueryService searchService =
    new SearchService.QueryService();
searchService.Credentials = CredentialCache.DefaultCredentials;

The code in the following section uses the searchService variable to reference the service proxy class. The code assumes that this variable is already set to an instance of the proxy class.

Searching with the Search Object Model

The following example code uses the KeywordQuery class when using the query object model to perform keyword searches. The code shows how to create and initialize an instance of the KeywordQuery class.

SPSite site = new SPSite("http://moss.litwareinc.com");
KeywordQuery kwQuery = new KeywordQuery(site);           
kwQuery.ResultTypes = ResultType.RelevantResults;

The KeywordQuery constructor is overloaded to accept an SSP context, SharePoint SPSite object, or a SharePoint application name. The code passes in an SPSite object for the top-level site. To run the code in your environment, you must change the site URL to match your environment. The code also sets the ResultTypes property of the KeywordQuery class to ResultType.RelevantResults to specify that the query results set contains the main search results.

The following e code uses the kwQuery variable and assumes that this variable is already set to an instance of the KeywordQuery class.

Filtering with Scopes

You can filter and focus search queries through the use of scopes. In Creating the Search Scope, you created a search scope named Word Documents that was associated with a document library. The following code performs a query for the term "Lorem" by using the Query Web service and limiting the search to the Word Documents scope.

StringBuilder queryXML = new StringBuilder();
queryXML.Append("<QueryPacket xmlns='urn:Microsoft.Search.Query'>");
queryXML.Append("<Query>");
queryXML.Append("<SupportedFormats>");
queryXML.Append("<Format revision='1'>");
queryXML.Append("urn:Microsoft.Search.Response.Document:Document");
queryXML.Append("</Format>");
queryXML.Append("</SupportedFormats>");
queryXML.Append("<Context>");

// Set the type attribute of the QueryText element to "STRING" to
// specify that we're doing a keyword query.
queryXML.Append("<QueryText language='en-US' type='STRING'>");

// Search for the term "Lorem" and limit the search to the scope
// named "Word Documents".
queryXML.Append(@"Lorem scope:""Word Documents""");

queryXML.Append("</QueryText>");
queryXML.Append("</Context>>");
queryXML.Append("<Properties>");
queryXML.Append("<Property name=\"Title\"></Property>");
queryXML.Append("<Property name=\"Path\"></Property>");
queryXML.Append("<Property name=\"Description\"></Property>");
queryXML.Append("<Property name=\"Write\"></Property>");
queryXML.Append("<Property name=\"Rank\"></Property>");
queryXML.Append("<Property name=\"Size\"></Property>");
queryXML.Append("<Property name=\"CollapsingStatus\"></Property>");
queryXML.Append("</Properties>");
queryXML.Append("</Query>");
queryXML.Append("</QueryPacket>");

string resultXML = searchService.Query(queryXML.ToString());

The code uses the StringBuilder class to build a query based on the Microsoft.Search.Query schema for Enterprise Search. It then calls the Query method of the Query Web service to have the search results returned in a string as XML. The returned XML uses the Microsoft.Search.Response.Document schema. The Query Web service also provides the QueryEx method, which returns search results in a System.Data.DataSet object.

Following is the returned XML. The XML is based on the context created earlier (see Setting the Context), with the scope associated with the document library that contains two instances of the same document with the word "Lorem" in them.

<ResponsePacket xmlns="urn:Microsoft.Search.Response">
  <Response>
    <Range>
      <StartAt>1</StartAt>
      <Count>1</Count>
      <TotalAvailable>1</TotalAvailable>
      <Results>
        <Document xmlns="urn:Microsoft.Search.Response.Document">
          <Action>
            <LinkUrl fileExt="docx">http://moss.litwareinc.com/Word Documents/Test.docx</LinkUrl>
          </Action>
          <Properties xmlns="urn:Microsoft.Search.Response.Document.Document">
            <Property>
              <Name>Title</Name>
              <Type>String</Type>
              <Value>Test</Value>
            </Property>
            <Property>
              <Name>Path</Name>
              <Type>String</Type>
              <Value>http://moss.litwareinc.com/Word Documents/Test.docx</Value>
            </Property>
            <Property>
              <Name>Write</Name>
              <Type>DateTime</Type>
              <Value>2008-04-10T16:18:41-07:00</Value>
            </Property>
            <Property>
              <Name>Rank</Name>
              <Type>Int64</Type>
              <Value>657</Value>
            </Property>
            <Property>
              <Name>Size</Name>
              <Type>Int64</Type>
              <Value>16049</Value>
            </Property>
            <Property>
              <Name>CollapsingStatus</Name>
              <Type>Int64</Type>
              <Value>1</Value>
            </Property>
          </Properties>
        </Document>
      </Results>
    </Range>
    <Status>SUCCESS</Status>
  </Response>
</ResponsePacket>

The following code shows the same search by using the Query object model.

kwQuery.QueryText = @"Lorem scope:""Word Documents""";

ResultTableCollection resultTableCollection = kwQuery.Execute();
ResultTable relevantResults =
    resultTableCollection[ResultType.RelevantResults];

The code sets the QueryText property of the KeywordQuery object and then calls the object's Execute method. The Execute method returns the search results in a ResultsTableCollection object.

The search results XML returned by the Query Web service's Query method contained only one document. However, the environment you created earlier had two documents with the word "Lorem" in them. Why weren't both documents returned in the search results? The answer has to do with the duplicates detection that is performed by Enterprise Search.

Working with Duplicate Results

In the previous example, only one document was returned in your search results, but there are two documents that should have matched the search term "Lorem". Duplicates filtering is the default search mechanism, and it caused this result. In this case, Enterprise Search determined that the two documents in your document library were actually the same document, so it returned only one of them. However, it informed you that the search results for the document were collapsed. The XML returned by the Query Web service specifies a value of 1 for the CollapsingStatus property of the returned document.

<Property>
  <Name>CollapsingStatus</Name>
  <Type>Int64</Type>
  <Value>1</Value>
</Property>

When querying via the object model, you can check for duplicates collapsing by looking at the value of the CollapsingStatus property on the results in the ResultTable object.

ResultTableCollection resultTableCollection = kwQuery.Execute();
ResultTable relevantResults = 
  resultTableCollection[ResultType.RelevantResults];

while (relevantResults.Read())
{
  if (Convert.ToInt32(relevantResults["CollapsingStatus"]) ==
    (int)(CollapsingStatus.DuplicatesCollapsed))
  {
    // There are duplicates for this item.
  }
}

If the value of the CollapsingStatus property is set to CollapsingStatus.DuplicatesCollapsed, you can perform another query for the document and request duplicates by using the duplicates filter. The following code queries for the document for which search results were collapsed, and then requests duplicates in the search results.

// Search for the term "Lorem" requesting duplicates for the
// specified document.
queryXML.Append("Lorem ");
queryXML.Append(
@"duplicates:""http://moss.litwareinc.com/Word Documents/Test.docx""");

The following code shows the same query by using the Query object model.

kwQuery.QueryText = "Lorem " +
 @"duplicates:""http://moss.litwareinc.com/Word Documents/test.docx""";

You must specify the original search term of "Lorem" even though you are specifying the full URL to the document you want duplicates for. If you do not, you get a result of ERROR_NO_RESULTS_FOUND. This result is returned because Enterprise Search needs the original query to return duplicates results that are consistent with that original query for items such as summary text and hit highlighting. The SharePoint View Duplicates page uses this functionality when you click the View Duplicates link for an item on the search results page.

Duplicates collapsing is the default behavior. You can change the default behavior by using the TrimDuplicates element with the Query Web service.

queryXML.Append("<TrimDuplicates>false</TrimDuplicates>");

You can control duplicates collapsing when using the Query object model by setting the TrimDuplicates property of the KeywordQuery class.

kwQuery.TrimDuplicates = false;

Filtering with Managed Properties

You can filter and focus search queries by using managed properties. In Creating the Managed Property, you created a managed property named WordDocumentsColumn1 that was mapped to a custom column in a document library. The following code shows how to perform a search with the Query Web service and specify a filter by using the WordDocumentsColumn1 managed property.

StringBuilder queryXML = new StringBuilder();
queryXML.Append("<QueryPacket xmlns='urn:Microsoft.Search.Query'>");
queryXML.Append("<Query>");
queryXML.Append("<SupportedFormats>");
queryXML.Append("<Format revision='1'>");
queryXML.Append("urn:Microsoft.Search.Response.Document:Document");
queryXML.Append("</Format>");
queryXML.Append("</SupportedFormats>");
queryXML.Append("<Context>");

// Set the type attribute of the QueryText element to "STRING" to
// specify that we're doing a keyword query.
queryXML.Append("<QueryText language='en-US' type='STRING'>");

// Search for the term "Test" filtering by the WordDocumentsColumn1
// managed property.
queryXML.Append(@"WordDocumentsColumn1:""Test""");

queryXML.Append("</QueryText>");
queryXML.Append("</Context>>");
queryXML.Append("<Properties>");
queryXML.Append("<Property name=\"Title\"></Property>");
queryXML.Append("<Property name=\"Path\"></Property>");
queryXML.Append("<Property name=\"Description\"></Property>");
queryXML.Append("<Property name=\"Write\"></Property>");
queryXML.Append("<Property name=\"Rank\"></Property>");
queryXML.Append("<Property name=\"Size\"></Property>");
queryXML.Append("<Property name=\"CollapsingStatus\"></Property>");
queryXML.Append("</Properties>");
queryXML.Append("</Query>");
queryXML.Append("</QueryPacket>");

string resultXML = searchService.Query(queryXML.ToString());

This code is very similar to the code that queried by using the scope filter. This time you are filtering with the WordDocumentsColumn1 managed property. The following code example shows the XML returned by the Query Web service Query method. You get back the one document that had its Column1 value set to a value that included the word "Test".

<ResponsePacket xmlns="urn:Microsoft.Search.Response">
  <Response>
    <Range>
      <StartAt>1</StartAt>
      <Count>1</Count>
      <TotalAvailable>1</TotalAvailable>
      <Results>
        <Document xmlns="urn:Microsoft.Search.Response.Document">
          <Action>
            <LinkUrl fileExt="docx">http://moss.litwareinc.com/Word Documents/Test.docx</LinkUrl>
          </Action>
          <Properties xmlns="urn:Microsoft.Search.Response.Document.Document">
            <Property>
              <Name>Title</Name>
              <Type>String</Type>
              <Value>Test</Value>
            </Property>
            <Property>
              <Name>Path</Name>
              <Type>String</Type>
              <Value>http://moss.litwareinc.com/Word Documents/Test.docx</Value>
            </Property>
            <Property>
              <Name>Write</Name>
              <Type>DateTime</Type>
              <Value>2008-04-10T16:18:41-07:00</Value>
            </Property>
            <Property>
              <Name>Rank</Name>
              <Type>Int64</Type>
              <Value>1000</Value>
            </Property>
            <Property>
              <Name>Size</Name>
              <Type>Int64</Type>
              <Value>16049</Value>
            </Property>
            <Property>
              <Name>CollapsingStatus</Name>
              <Type>Int64</Type>
              <Value>0</Value>
            </Property>
          </Properties>
        </Document>
      </Results>
    </Range>
    <Status>SUCCESS</Status>
  </Response>
</ResponsePacket>

The following code example shows the same query by using the Query object model.

kwQuery.QueryText = @"WordDocumentsColumn1:""Test""";

Conclusion

The keyword query syntax supported by Enterprise Search in Microsoft Office SharePoint Server makes creating simple search queries easy. It allows you to pass search terms directly to the search service. Property filters allow you to use search scopes and managed properties to focus your searches and to handle duplicates results collapsing.

Additional Resources

For more information, see the following resources: