Eric Dixon's Search Blog https://docs.microsoft.com/archive/blogs/ericdixon/feed.xml My blog covers search related topics for the SharePoint Online, SharePoint 2016, SharePoint 2013, FASTSearch for SharePoint 2010, and FAST ESP search engines. Tue, 16 Apr 2019 05:07:04 GMT en-US hourly 1 Intelligent Search: Adding AI to SharePoint Search https://docs.microsoft.com/archive/blogs/ericdixon/adding-ai-to-sharepoint-search Thu, 19 Apr 2018 16:55:39 GMT https://blogs.msdn.microsoft.com/ericdixon/?p=125 Cognitive Services, and many of these can help search on both the content and query side of SharePoint search. My team and I have been working on an AI Query Pipeline.  There are many benefits to adding AI to the query.  I'm going to show examples of four of them. They are:
  • Contextual and phonetic spellchecking
  • Translation
  • Query intent
  • NLP parsing
  In this example, the query is "markting slid".  The first term is misspelled, which is a very easy correction.  Where AI kicks in is on the second term.  Technically, it is a term which is spelled correctly, but contextually it is wrong. Phonetic spell correction. Here is an example of translating Chinese to English and getting results for both. NLP processing of queries.  What's not seen is that on the query, since the personal pronoun 'my' was included, filters were added to the query to search 'my' stuff like 'me' as the author or 'my' OneDrive. Combining translation and NLP Deriving intent from similar queries  ]]> SharePoint Search: – “Why are my results so bad?” Part 2 - PDFs vs. Office Documents https://docs.microsoft.com/archive/blogs/ericdixon/sharepoint-search-why-are-my-results-so-bad-part-2-pdfs-vs-office-documents Fri, 21 Apr 2017 19:43:13 GMT https://blogs.msdn.microsoft.com/ericdixon/?p=96 previous post, I discussed why search results can seem so bad.  Honestly, labeling results as "bad" is misleading because the Search Engine is ranking results by only the metadata and rank model that the Search Admin told the Search Engine to use.  In reality, there are no bad search results, only bad Search Admins. ;) I recently ran across a case of "perceived bad results" that I thought was really interesting because the ranking behavior from the default SP2013 rank model had completely flipped when compared to the ranking behavior in SPO.  In SP2013, customers often questioned why, when given an Office Document (e.g. PowerPoint or Word) and a PDF which were equivalent in content, the Office Document always ranked higher. The reason is that the SP2013 default rank model heavily weighs PowerPoint and Word file types while PDFs are not weighted.  In fact, in the SP2013 default rank model, Office Documents are the most heavily weighted of all document types.  Since a PDF is not weighted in the default rank model, it receives a default weighting that is lower than Office Documents' weightings. In SPO, this behavior has "flipped," not because PDFs are weighted higher, but because Office Documents are weighted lower than the default File Type, and since PDFs have no defined weighting, they will get the default or higher weighting. Adding to the foregoing confusion is this question: "What defines the default weight for a File Type?"  During indexing, all files are assigned an "Internal File Type."  For example, a Word document is InternalFileType=1, a PowerPoint is InternalFileType=2, and a PDF is InternalFileType=15.  While the default rank model in SPO and SP2013 define weights for Types 1 and 2, they do not define a weight for 15.  During rank score calculation, a PDF's File Type is transformed from 15 to 0.  This transformation is clearly seen in the rank log output (see the Search Query Tool for more on how to get a rank log).  The default rank log does define a weighting for Type 0, but the rank models label this type as HTML.  So, in reality, any undefined File Type will receive the same weighting as an HTML file and in SPO it will be higher than Office Documents, while in SP2013 it will be lower. 8-O Below is a excerpt from a rank log using the default SPO rank model of a PowerPoint document showing a the InternalFileType=2 (PPT) getting a boost of -0.078.
<bucketed_static_feature name='InternalFileType' property_name='InternalFileType' used_default='0' raw_value_in='2' raw_value='2' raw_value_transformed='2' hidden_nodes_adds='-0.0780378'/>
Compare to the following excerpt from a rank log using the default SPO rank model of a PDF document showing an initial InternalFileType=15 (PDF), then transforming the value to 0 (HTML/default), and applying a boost of 0.391.
<bucketed_static_feature name='InternalFileType' property_name='InternalFileType' used_default='1' raw_value_in='15' raw_value='15' raw_value_transformed='0' hidden_nodes_adds='0.39136'/>
To address this issue in SP2013 or SPO, you should to define an appropriate weight for PDFs in a custom rank model or use the XRANK operator to apply an appropriate boost for PDFs.]]> SharePoint 2013/2016 Search: When standard architecture won't work for you https://docs.microsoft.com/archive/blogs/ericdixon/sharepoint-20132016-search-when-standard-architecture-wont-work-for-you Mon, 27 Feb 2017 20:14:51 GMT https://blogs.msdn.microsoft.com/ericdixon/?p=95 TechNet search architecture article that provides examples of architectures for small, medium, large, and extra-large search farms.  The answer to their question is always, "It depends."  What it depends on is whether you have "standard" or "non-standard" requirements. The example architectures provided in TechNet will work for most people, but some requirements or conditions may demand modifications to the examples TechNet provides.  The following question invariably arises: "What are non-standard search architecture requirements?" These are examples of non-standard requirements:
  • Above-average DPS
  • Above-average QPS
  • Above-average number of start addresses or having HNSC enabled
  • Bare-minimum hardware or over-subscribed hosts
  • Above-average indexed item size
  • Large Search Schema (Site Columns)
  Above-average DPS If you have requirements for an above-average Documents Per Second (DPS) or content-processing rate, you may need to add resources or dedicate servers to Content Processing Components (CPS).  CPCs need a lot of memory and CPU to push items through their processing pipelines. Above-average QPS If you have requirements for an above-average Queries Per Second (QPS) rate, you may need to add resources or dedicate servers to Query Processing and Index Components (QPC/IC).  QPCs and ICs both need physical cores to maintain QPS.  If other search components (or system processes) are competing for those cores, resource scarcity may cause QPS to be lower or latency to be higher. Above-average number of start addresses or having HNSC enabled If you have a high number of start addresses or have Host-Name Site Collections (HNSC) enabled, the crawler will spawn more threads to crawl each unique host name it encounters.  Crawler threads run at a higher priority than other search components' threads.  When lots of unique host names are being crawled, the number of threads the crawler uses may cause other search components to become resource starved.  Please refer to Brian Pendergrass's SharePoint Strategery blogs for all things crawl (and many other wonderful search nuggets). Bare-minimum hardware or over-subscribed hosts CPCs, QPCs and ICs all rely heavily on CPUs and their cores.  If you are running multiple components on servers with the minimum number of CPUs/cores, in heavy-load scenarios (high DPS or QPS), your search components will be competing with each other for resources and may suffer performance degradation.  ICs require server-grade hard drives that can support the IC's IOP requirements.  Host server hardware must adequately support VM demands.  In general, you should treat search components, especially indexers, like SQL servers.  Here is an overview of virtualization for SharePoint. Above-average indexed item size Standard guidance for an IC hard drive is 500 GB.  For SP 2013, guidance is based on an average indexed item size of 20KB (Not the original document size--the size of the indexed item on disk after processing).   For 2013, as you approach the max items per index partition (10 M), if your average is above 20KB, you risk running out of disk space when the indexer attempts to perform a master merge.  For 2016, running out of disk space is much less of a concern because the indexers have greatly reduced their disk footprint and master merge efficiency. Large Search Schema (Site Columns) A Site Column will auto-generate Managed Properties (MP) in the Search Schema.  Whether auto-generated or through customization, additional MPs impact the size of the Search Schema, which in turn impact an IC's index footprint on disk and memory.  How much will depend on which properties are selected on the MP (e.g. Searchable, Queryable, Refinable, Sortable).  With each MP, the number of mappings that occur during content processing and the size of the index will increase, thus increasing the computational load on CPCs.  If you require (or will generate) large schemas, you may need to dedicate additional resources to CPCs and ICs.   If you ever have questions on architecture requirements that may push your search system into the "non-standard" category, please contact your TAM and reach out to us at SearchEngineers@microsoft.com.  ]]> Relevant Relevancy Changes in SharePoint 2013 Sept and Oct 2016 CUs https://docs.microsoft.com/archive/blogs/ericdixon/relevant-relevancy-changes-in-sharepoint-2013-sept-and-oct-2016-cus Wed, 04 Jan 2017 17:37:00 GMT https://blogs.msdn.microsoft.com/ericdixon/?p=85 ResetSynonymWeightToOne and RecallContentOnlyInDefault. According to the CU notes, the flags enable functionality defined as follows: Sept. 2016 CU This update provides a ResetSynonymWeightToOne boolean flag in the Search Service Application, which can be set to $true to change the synonym weight to 100% of the original term. Oct. 2016 CU This update provides a new RecallContentOnlyInDefault boolean property in the search service application which can be set to $true to disable recall from all non-default update groups. Here is how these settings can impact you and scenarios under which you may want to enable them. ResetSynonymWeightToOne By default, the rank score contribution weight of a synonym is 20%.  For example, if you search using the query term "computer" and have a synonym for "computer" that is "laptop", the query is expanded.  The expanded query will match documents that contain "computer" or "laptop".  During ranking score calculation, the Term Frequency scores of "computer" and "laptop" will be calculated.  Since "laptop" is a synonym, its score will be reduced to 20% of its actual total, or in other words, multiplied by 0.2. This default behavior is often unexpected or undesired to customers who use synonyms and feel that synonyms should be considered equally when calculating ranking scores.  Setting the ResetSynonymWeightToOne to $true will change the weighting of synonyms to 100% or, in other words, the term frequency score is not discounted. Note: terms which expand queries through stemming or lemmatized forms are also weighted at 20%, but there currently is no supported way of changing this value. RecallContentOnlyInDefault This oddly named flag removes certain Managed Properties from consideration when recalling (or matching) documents for a query.  The Managed Properties not considered are those that are Searchable, but not in the default UpdateGroup.  To see a full list of all the properties that are affected you can run the following PowerShell: $ssa | Get-SPEnterpriseSearchMetadataManagedProperty | ? {$_.UpdateGroup -ne $null -and $_.Searchable} The most common scenario where you may want to consider using this flag is when the QLogClickedText Managed Property is populated.  The QLogClickedText property, which does not belong to the default UpdateGroup and is Searchable, is populated with the submitted query terms when a user clicks on a result.  For example, if a user submits the query "hardware" and clicks on search result #8 titled "A Computer Book", the QLogClickedText property for the document titled "A Computer Book" will be updated with the value "hardware". The intention is to tie query terms to documents users interact with to provide better relevancy.  Since, by default, the QLogClickedText property is used for recall (document matching) and relevancy (document sorting), it is possible to get query terms associated with documents which Search Admins and users not feel represent the document well or are considered irrelevant to the document.  In this case, setting the RecallContentOnlyInDefault to $true will not consider Managed Properties not in the default Update Group (e.g. QLogClickedText) when matching or recalling documents for a query.  The values in these properties continue to be used for relevancy ranking scores.  ]]> SharePoint 2013 Search: Even Better Best Bets with Exact Matching https://docs.microsoft.com/archive/blogs/ericdixon/sharepoint-2013-search-even-better-best-bets-with-exact-matching Fri, 23 Oct 2015 12:57:00 GMT https://blogs.msdn.microsoft.com/ericdixon/2015/10/23/sharepoint-2013-search-even-better-best-bets-with-exact-matching/ First, full credit goes to Mikael Svenson and Petter Skodvin-Hvammen for the original solution.  This blog details a small modification to Mikael's "Better Best Bets" blog found here: https://blogs.msdn.com/b/mvpawardprogram/archive/2014/07/21/better-best-bets-with-lists-in-sharepoint-2013.aspx.

Today's blog is a bare-bones, step-by-step process modifying the Better Best Bets solution to do exact matches on keywords.  The "Better Best Bets" solution can only do "contains" matches on keywords since the keyword column type is a taxonomy and has taxIds in the indexed content preventing exact matches.   

From the "Better Best Bets" blog:

If you have multi-word trigger terms, make sure all words are pretty good/unique to avoid unwanted partial matches. Or stick with the out of the box query rule matching with a promoted result instead.

 From <https://blogs.msdn.com/b/mvpawardprogram/archive/2014/07/21/better-best-bets-with-lists-in-sharepoint-2013.aspx>

For example, if you have the Best Bet Keywords "United", "United States" and "Manchester United", the contains operator and the keyword "United" will match all three.  This may be undesired behavior.  Having the ability to use the exact match operator, the Keyword "United" will only match the "United" Best Bet Keyword.

The steps below detail how to use SharePoint lists to manage best bets which will do exact matches on multi-value keywords.  For simplicity, the example below doesn't included the Date properties like the "Better Best Bets" solution, but they can be added and used just like they were previously.

This solutions uses two SharePoint lists.  The first list contains all the keywords.  The second list has a lookup column using the keywords in the first list.  This is required so multi-valued data will be preserved during content processing and the exact match operator will match the multi-valued keywords correctly.

From a non-search site create a List that will be a source for the individual keywords.

  1. Create Site Column for Keyword Source List
    1. Site Settings > Site Columns > Create
      1. Name: ebbbKeywordSource
      2. Type: Single line of text
      3. Put this site column into:
        1. New Group: Even Better Best Bets
      4. Require that this column…: Yes
      5. Enforce unique values: Yes
      6. OK
  2. Create Content Type for Keyword Source List
    1. Site Settings > Site content types > Create
      1. Name: EBBBKeywordSource
      2. Select parent content type…: List Content Types
      3. Parent Content Type: Item
      4. New Group: Even Better Best Bets
      5. OK
    2. Site Content Types > Site Content Type (Even Better Best Bets Keyword Source)
      1. Select Title column
        1. Select Hidden
        2. OK
      2. Add from existing site columns
        1. Select ebbbKeywordSource
        2. Click Add
        3. OK
  3. Create a Custom Keyword Source List
    1. List name: Even Better Best Bets Keyword Source List
    2. Select List (Even Better Best Bets Keyword Source List)
      1. List tab > List Settings
      2. Advanced Settings
        1. Allow management of content types: Yes
        2. Allow items from this list to appear in search results: No
        3. OK
      3. Add from existing site content types
        1. Select EBBBKeywordSource
        2. Click Add
        3. OK
    3. Click Change new button order and default content type
      1. Content Type: Item
        1. Uncheck Visible
      2. Content Type: EvenBetterBestBets
        1. Position from Top: 1
      3. OK
    4. Views > All Items
      1. Title
        1. Uncheck Display
      2. ebbbKeywordSource
        1. Check Display
      3. OK
    5. Enter Keyword values in ebbbKeywordSource column
      1. Click "new item"
        1. Enter a unique word or phrase
        2. Save
      2. Repeat

 

From a non-search site create a List that will reference the Keywords Source List and hold the data used to create the Best Bets.

  1. Create Site Columns
    1. Site Settings > Site Columns > Create
      1. Name: ebbbTitle
      2. Type: Single line of text
      3. Put this site column into:
        1. Existing Group: Even Better Best Bets
      4. Require that this column…: Yes
      5. Enforce unique values: Yes
      6. OK
    2. Site Settings > Site Columns > Create
      1. Name: ebbbURL
      2. Type: Single line of text
      3. Put this site column into:
        1. Existing Group: Even Better Best Bets
      4. Require that this column…: Yes
      5. OK
    3. Site Settings > Site Columns > Create
      1. Name: ebbbKeywordsMatch
      2. Type: Lookup
      3. Put this site column into:
        1. Existing Group: Even Better Best Bets
      4. "Require that this column…": Yes
      5. Get information from: Even Better Best Bets Keyword Source List
      6. In this column: ebbbKeywordSource
        1. Allow multiple values: Yes
  2. Create Content Type
    1. Site Settings > Site content types > Create
      1. Name: EBBBList
      2. Select parent content type…: List Content Types
      3. Parent Content Type: Item
      4. New Group: Even Better Best Bets
      5. OK
    2. Site Content Types > Site Content Type (EBBBList)
      1. Click Title column
        1. Select Hidden
        2. OK
      2. Add from existing site columns
        1. Select ebbbKeywordsMatch, ebbbTitle, ebbbURL
        2. Click Add
        3. OK
  3. Create Best Bets List
    1. List name: Even Better Best Bets List
    2. Create
    3. Select List (Even Better Best Bets List)
      1. List tab > List Settings
      2. Advanced settings > Allow management of content types: Yes
      3. OK
      4. Add from existing site content types
        1. Select EBBBList
        2. Click Add
        3. OK
      5. Click Change new button order and default content type
        1. Content Type: Item
          1. Uncheck Visible
        2. Content Type: EBBBList
          1. Position from Top: 1
      6. Views > All Items
        1. Title
          1. Uncheck Display
        2. ebbbKeywordsMatch, ebbbTitle, ebbbURL
          1. Check Display
        3. OK
  4. Enter data for Even Better Best Bets List
    1. Click "new item"
    2. Add data for ebbbKeywordsMatch, ebbbTitle and ebbbURL
    3. Save
    4. Repeat
  5. SSA Admin > Content Sources > Local SharePoint sites > Full Crawl  (or Content Source pointing to your site)
    1. Once crawl has completed, the query spcontenttype:ebbbList will return all items in Even Better Best Bets List
  6. SSA Admin > Search Schema > Crawled Properties (or at the desired Tenant level)
    1. Search for ebbb
    2. For each ows_ebbb* field
      1. Uncheck "Include in full-text index"
  7. SSA Admin > Search Schema > Managed Properties (or at the desired Tenant level)
    1. Verify fields ebb*OWSTEXT exists for Title and URL with Query and Retrieve properties (not Search) with a mapping.
    2. New Managed Property
      1. Name: ebbbKeywords
      2. Type: Text
      3. Check Queryable
      4. Check Retrievable (optional, good for debugging)
      5. Check Allow multiple values
      6. Add a Mapping
        1. Select ows_ebbbKeywordsMatch
        2. OK
      7. OK
  8. SSA Admin > Content Sources > Local SharePoint sites > Full Crawl  (or Content Source pointing to your site)
    1. Once crawl has completed, the query ebbbKeywords="<keywords>"spcontenttype:ebbbList will return all items in Even Better Best Bets List that match <keywords>.  Supports exact match on multivalues.

 

From the Search Center Site

  1. Create Display Template
    1. Search Center > Site Settings > Master pages and page layouts > Display Templates > Search > Item_BestBets.html > Download a Copy > Save as Item_EvenBetterBestBets.html.
    2. Edit Item_EvenBetterBestBets.html
      1. <title>Even Better Best Bets</title>
      2. <mso:ManagedPropertyMapping msdt:dt="string">'bbTitleOWSTEXT':'bbTitleOWSTEXT','bbURLOWSTEXT':'bbURLOWSTEXT'</mso:ManagedPropertyMapping>
      3. Save
    3. Display Templates > File Tab > Upload Document
      1. Select file
      2. OK
      3. Save
  2. Create Result Type
    1. Search Center > Site Settings > Site Collection Administration > Search Result Types > New Result Type
      1. Name: Even Better Best Bets
      2. Show more conditions
        1. Which custom properties should match: Content Type
        2. (Value): EBBBList
      3. What should these results look like: Even Better Best Bet Item
      4. Check Optimize for frequent use
      5. Save
  3. Create Result Source
    1. Search Center > Site Settings > Site Collection Administration > Search Result Sources > New Result Source
      1. Name: Even Better Best Bets
      2. Query Transform: ebbbKeywords="{searchTerms}" spcontenttype=ebbblist
      3. Save
  4. Create Query Rule
    1. Search Center > Site Settings > Site Collection Administration > Search Query Rules
      1. Select "Local SharePoint Results" > New Query Rule
        1. Rule Name: Even Better Best Bets
        2. Remove Condition
        3. Add Result Block
          1. Title: Best Bets for "{subject Terms}"
          2. Search this Source: Even Better Best Bets
          3. Items: 5
          4. Click Settings
            1. Select This block is always shown above core results
          5. OK
        4. Change ranked results by changing the query (hide best bets content and list AllItems.aspx from "default" search results).
          1. Query Text: {searchTerms} -spcontenttype:ebbblist -ListURL:"Even Better Best Bets List"
          2. OK
        5. Save
  5. Test Best Bets from Search Box

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


 

 

 

]]> SharePoint 2013 Search: How clicks change relevancy and rank scoring https://docs.microsoft.com/archive/blogs/ericdixon/sharepoint-2013-search-how-clicks-change-relevancy-and-rank-scoring Thu, 03 Sep 2015 11:00:00 GMT https://blogs.msdn.microsoft.com/ericdixon/2015/09/03/sharepoint-2013-search-how-clicks-change-relevancy-and-rank-scoring/ If you've ever looked at rank log or the ExplainRank.aspx page, you may have noticed that term frequency scoring is only a portion of the total rank score.  There are many other relevancy features which contribute to rank scoring.  Among them are "click events".  Click events measure the interaction, or lack of interaction, between a user and the search results.  The information is then used to boost or de-boost documents accordingly.  There are four "click events" which contribute to relevancy.  They are, as named in the ExplainRank.aspx page: QueryLogClicks, QueryLogSkips, LastClicks and EventRate.  The rank log, respectively, names them QLogClicks, QLogSkips, QLogLastClicks and EventRate.  There is another event named ClickDistance, but technically it has nothing to do with click events.  ClickDistance is a boost based on the distance or number of links (could be thought of as clicks) between a document and an authoritative source.
Here are definitions for the four click event relevancy features.
  • QueryLogClicks - A boost based on how often a document is clicked on in search results per query.
  • QueryLogSkips - A de-boost based on how often another document of a lower rank is clicked on, instead of this document, in search results per query.
  • LastClicks - A freshness like boost based on the last time this document was clicked on in a search result.
  • EventRate - A boost based on how often a SharePoint document is clicked or viewed (not in search results). 
Note: EventRate is only tracked automatically for SharePoint documents.  If desired, you can manually track events on external content through the .NET or JavaScript SearchExecutor class.  Here are some links if you are interested in more details.
]]> SharePoint 2013 Search: Full-Text Index and Context Weight Groups - What is wrong with my rank model? https://docs.microsoft.com/archive/blogs/ericdixon/sharepoint-2013-search-full-text-index-and-context-weight-groups-what-is-wrong-with-my-rank-model Mon, 10 Aug 2015 16:08:00 GMT https://blogs.msdn.microsoft.com/ericdixon/2015/08/10/sharepoint-2013-search-full-text-index-and-context-weight-groups-what-is-wrong-with-my-rank-model/ blog, I talked about the important relationship between relevancy, rank models, managed properties, the full-text index (FTI) and context weight groups.  If you are going to doing any relevancy tuning, you must understand how managed properties map to context weight groups in the full-text index and how, through the rank model, you can assign weights to context weight groups by referencing a managed property belonging to a group. Here is a TechNet article which gives a simple overview of that relationship. What the article leaves out is that in the Default FTI, for some contexts, in addition to Context 0, it is possible to encounter scenarios where contexts will not contribute to Term Frequency scoring. For example, if a Query has hits ONLY in Context 3, the Term Score will not be calculated.  If you added managed properties to Context 3, hits on terms in those managed properties will only contribute to recall.  In the screenshot from the ExplainRank.aspx page below, you see that despite the query term 'update' being present in Context 3 (TF=1) the Weighted TF and Term Score is 0.
When adding a Managed Property to a Context Weight Group in a Full-Text Index, if you want a term score calculated, be aware when using the following Contexts: 0, 3, 6, 11 and 14.  If the Query matches ONLY one of the previously listed Contexts, the Term Score will NOT be calculated and the term hits in the group will only contribute to recall.  It's probably best to not use these Contexts all together when adding custom Managed Properties to a Context.
When adding a managed property that should contribute to a term score, first choose from contexts that have data similar in importance and length (check out the 'b' value in the rank model).  If you want the managed property to always contribute to the term score, stick to Context 1 (Title), Context 2 (Filename), Context 5 (Author), Context 7 (Body) or use an "unassigned" context like 9, 10, 12, 13 (I had originally listed Context 4 (Keywords) as a recommend context, but TechNet documentation does not call it out specifically like the others).  I've tested these contexts and relevancy works correctly.  You might have success with other contexts, but you need to thoroughly test it to ensure the term score is calculated as expected.]]> SharePoint 2013 Search: People Search – “Why are my results so bad?” Understanding Relevancy, the Rank Model, Full-Text Index, Fuzzy Matching, and Social Distance https://docs.microsoft.com/archive/blogs/ericdixon/sharepoint-2013-search-people-search-why-are-my-results-so-bad-understanding-relevancy-the-rank-model-full-text-index-fuzzy-matching-and-social-distance Wed, 03 Jun 2015 11:16:00 GMT https://blogs.msdn.microsoft.com/ericdixon/2015/06/03/sharepoint-2013-search-people-search-why-are-my-results-so-bad-understanding-relevancy-the-rank-model-full-text-index-fuzzy-matching-and-social-distance/ Recall and Precision In the context of information retrieval, or search, Recall and Precision measure the completeness and accuracy of a query’s results.  For our purposes, Recall defines the set of all possible documents that match a query, and Precision is the sort of the recalled set by a relevancy score.  In SharePoint 2013, we can equate Recall to the Full-Text Index and Precision to the Ranking Model. Full-Text Index In SharePoint 2013, when users create a Managed Property and set the Searchable property to “true”, they have the opportunity to assign the property to a Full-Text Index in the Advanced Searchable Settings.  SharePoint 2013 has several Full-Text Indexes, but only two are used by default:
  1. Default—the Full-Text Index for the "Local SharePoint Results" result source or the “Everything” vertical
  2. PeopleIdx— the Full-Text Index for the “Local People Results” result source or the “People” vertical.  All the people relevant managed properties are assigned to the PeopleIdx Full-Text Index
Some simple PowerShell reveals all Managed Properties assigned to PeopleIdx:

> Get-SPEnterpriseSearchMetadataManagedProperty -SearchApplication $ssa | ? {$_.FullTextIndex -eq "PeopleIdx"} | ft Name, FullTextIndex, Context

Name                                    FullTextIndex                                                           Context

----                                    -------------                                                           -------

AccountName                             PeopleIdx                                                                     9

MyFirstName                           PeopleIdx                                                                    15

MyLastName                            PeopleIdx                                                                    15

CombinedName                            PeopleIdx                                                                    14

ContentsHidden                          PeopleIdx                                                                     5

JobTitle                                PeopleIdx                                                                     2

Memberships                             PeopleIdx                                                                     6

NgramPhoneNumbers                       PeopleIdx                                                                     0

PreferredName                           PeopleIdx                                                                     1

Pronunciations                          PeopleIdx                                                                     8

RankingWeightHigh                       PeopleIdx                                                                     7

RankingWeightLow                        PeopleIdx                                                                     4

RankingWeightName                       PeopleIdx                                                                    13

Responsibilities                        PeopleIdx                                                                     3

SipAddress                              PeopleIdx                                                                    12

UserName                                PeopleIdx                                                                    10

WorkEmail                               PeopleIdx                                                                    11

  These are all the Managed Properties that will be used for Recall when doing a people search.  There is a loose coupling between a Full-Text Index and a Rank Model.  Note that a Rank Model may not show all of the Managed Properties assigned to a Full-Text Index, so don’t forget that, despite their absence, all Managed Properties in the Full-Text Index contribute to recall, but with no weighting.  Also, any Managed Property that shares the same context as a property weighted in a Rank Model will receive the same weighting during rank score calculation.  So, if you add a Managed Property named Interests and assign it to context 3, and the Rank Model weights Responsibilities (e.g., w=”1.0”), Interests will receive the same weighting, despite not being specifically named in the Rank Model. People Search Rank Models The default People Search Rank Model is named “People Search application ranking model” (https://technet.microsoft.com/library/7c8ddec1-c8ff-4a90-afae-387b27a653f1.aspx#Ranking_Models).  Rank Models provide Precision or Relevancy Ranking for a recalled search result set.  Note that of the 16 OOB rank models, six are specific to People Search.  One of these rank models may serve your needs better than the default.  To choose which model is best for you, you’ll need to understanding the rank model beyond its description and understand the configuration of the rank model. PowerShell provides some nifty Cmdlets that export the rank model configuration XML (https://msdn.microsoft.com/en-us/library/office/dn169052(v=office.15).aspx#sp15_using_custom_ranking_model ).  If you look at the XML config for the default People Search ranking model, you should notice a couple things.  First, the model has two stages (<RankingModel2NN>).  The first stage is a lighter calculation used to sort all the results that were matched or recalled from the search index.  The second stage is used to resort the top 1000 results.  It uses a more complex sorting algorithm, adding proximity weights to the calculation.  Not all ranking models have two stages; some models (e.g. People Search name ranking model) only have one stage.  Second, the ranking model names all the fields that will be weighted with their weight values and a bias value. Here is an excerpt of the properties definition and weightings from the default people search rank model.

       <Properties>

          <Property name="RankingWeightName" w="0.5" b="0.5" propertyName="RankingWeightName" />

          <Property name="PreferredName" w="1.0" b="0.5" extractOccurrence="1" propertyName="PreferredName" />

          <Property name="JobTitle" w="2.0" b="0.5" extractOccurrence="1" propertyName="JobTitle" />

          <Property name="Responsibilities" w="1.0" b="0.85714285719" extractOccurrence="1" propertyName="Responsibilities" />

          <Property name="RankingWeightLow" w="0.2" b="0.85714285719" extractOccurrence="1" propertyName="RankingWeightLow" />

          <Property name="ContentsHidden" w="0.1" b="0.85714285719" extractOccurrence="1" propertyName="ContentsHidden" />

          <Property name="Memberships" w="0.25" b="0.85714285719" extractOccurrence="1" propertyName="Memberships" />

          <Property name="RankingWeightHigh" w="2.0" b="0.5" extractOccurrence="1" propertyName="RankingWeightHigh" />

          <Property name="Pronunciations" w="0.05" b="0.5" propertyName="Pronunciations" />

        </Properties>

 Social Distance

An important note needs to be made here about Social Distance Ranking.  As you explore the default people ranking model, you will notice dynamic ranking features such as FirstLevelColleagues, SecondLevelColleagues and LevelsToTop.  These are part of the Social Distance ranking feature, and their intention is to add a relevancy boost to colleagues and management close to you, with respect to your company's organizational chart.  The issue is that, out of the box, it doesn't work correctly.**  I've heard conflicting reports that recent patches have fixed this issue for both SPO and On-prem SharePoint environments.  I have not been able to verify.  Please test thoroughly.  If it is not working, an examination of the ULS log for a "default" People search query will reveal the following entry:

07/17/2014 01:13:34.85  NodeRunnerQuery1-963b71f2-33dc- (0x1280)    0x1584  Search  Query Processing             aiziq                Monitorable                Microsoft.Office.Server.Search.Query.Pipeline.Processing.PersonalizationDataInjectionEvaluator : Field: PersonalizationData 'null'. Personalized Search queries will not work       0be6b1d5-3bd8-4aac-a1f1-08d7c75867cc

The web part is not setting the "PersonalizationData" parameter.  The consequence is the null parameter prevents Social Distance Ranking from being calculated.  Fortunately, there are three workarounds, but the last is the most simple to implement. For all workarounds, verify the Web Application hosting the Enterprise Search center is associated with a UPSA proxy (This association should work OOB with a 2013 proxy, but not 2010). (1) Use the Query REST API and add the "PersonalizationData" parameter with the search user's User Profile GUID as the value.  (2) Create a custom search web part and add the User Profile GUID.  SCOM uses the parameter name "QueryPersonalizationData".  (3) Edit the People Search results page, edit the People Search Core Results web part, and click Change Query:

Old:

{searchboxquery} New: {?(({searchboxquery} XRANK(cb=2) firstlevelcolleagues:{User.userprofile_guid}) XRANK(cb=1) secondlevelcolleagues:{User.userprofile_guid}) XRANK(cb=7.5) userprofile_guid:{User.userprofile_guid}} Or if you want to get real fancy:

 {?((((((({searchboxquery} XRANK(cb=2) firstlevelcolleagues:{User.userprofile_guid}) XRANK(cb=1) secondlevelcolleagues:{User.userprofile_guid}) XRANK(cb=7.5) userprofile_guid:{User.userprofile_guid}) XRANK(cb=1) levelstotop:1) XRANK(cb=0.8) levelstotop:2) XRANK(cb=0.6) levelstotop:3) XRANK(cb=0.4) levelstotop:4) XRANK(cb=0.2) levelstotop:5}

Note that the XRANK operator can have an impact on query latency, so test this (or any solution) before putting it into production.  Also, note that latency is directly tied to index size, so what performs well in a small test environment may perform poorly in a very large production environment. **A big thanks to Mikkel Conradi for doing all the leg work on this issue. “Why are my results so bad?”  Back to the Katherine Doe/Cathy Smith scenario where searching for Katherine Doe produces results with Cathy Smith at the top. In reality, the results aren’t bad.  They are exactly what they should be.  Understanding the metadata and how it’s used in the rank model and Full-Text Index will explain why.  First, some background for our scenario.  Katherine Doe is an Executive and Cathy Smith is her Assistant.  In Cathy Smith’s User Profile under Responsibilities it lists “Executive Assistant for Katherine Doe”, while under Memberships, it shows Cathy is a member of “Katherine Doe Direct Reports” group. If we refer to the default People Rank Model, we see that it includes, along with PreferredName, Memberships and Responsibilities.  So a search for Katherine Doe produces hits in Cathy Smith’s Memberships and Responsibilities Managed Properties, while no hits are in the corresponding Managed Properties for Katherine Doe (Katherine’s name does not appear in these fields.  She is not a member of her direct reports group and she is not her own Executive Assistant).  The hits in the non-name fields are often forgotten or not noticed because they are not displayed in the default Item_Person.html display template and are only partially displayed in the Item_Person_HoverPanel.html display template.  Katherine does get a hit, for both first and last name, in the PreferredName field, but so does Cathy, at least on the first name.  Cathy is a nickname for Katherine, so Fuzzy Matching, which is enabled by default on People Searches, will add “Cathy” as a nickname synonym, for the Katherine term to the query. Fuzzy Matching To be honest, the documentation on the Fuzzy Matching feature in SharePoint 2013 is quite… fuzzy.  Often phonetic matching is mentioned when talking about SP 2013 Fuzzy Matching components, but to be clear, there is no intrinsic phonetic matching in SP 2013.  It did exist in SharePoint 2010, but it is not in SP 2013. SharePoint 2013 Fuzzy Matching is comprised of three services: (1) Core Fuzzy Name Search, (2) Name Suggestions, and (3) Name Intent Search.  The latter, Name Intent Search, is only used as part of a Query Rule condition used to identify People Queries from a “non-People” vertical.  Core Fuzzy Name Search is a spelling or distance algorithm that measures similarity of terms.  Results from this service can look a lot like a phonetic match, but they’re not.  For example, the query 'katherine' will return results for what appears to be its phonetic match 'catherine'.  Examination of verbose ULS logs reveals the real story.  The term 'catherine' is added to the query because of its spelling similarity or distance from 'katherine', not for its phonetic equivalency. Additionally, not only is 'catherine' considered, but so are 'gathering', 'katharine'.

06/04/2015 12:14:49.21 NodeRunnerQuery1-aaaaedc2-86d0- (0x0F24) 0x129C Search Fuzzy Name Search ajyg0 VerboseEx FuzzyNameSearcher : CoreCCAFuzzySearcher mined [Candidate:catherine GeometricSimilarity: 0.999740619082072 NormalizedConfidence:0.999740619082072] for Query:katherine IsComparable:true  531f0d9d-9c5c-c0a8-27b6-5ef65f3bbadb

06/04/2015 12:14:50.58 NodeRunnerQuery1-aaaaedc2-86d0- (0x0F24) 0x12B4 Search Query Processing aizf6 High Microsoft.Office.Server.Search.Query.Pipeline.Executors.LinguisticQueryProcessingExecutor : QSC: All Annotations: <Annotation ID="1" Name="token" Range="[0,9)" Attributes={normalizedForm="katherine"} NumericalAttributes={}/>,<Annotation ID="2" Name="querysegment" Range="[0,9)" Attributes={} NumericalAttributes={}/>,<Annotation ID="2001" Name="spellcheck" Range="[0,9)" Attributes={Fuzzy="catherine",Score="0.888888888888889",dynamic="",value="5"} NumericalAttributes={}/>,<Annotation ID="2002" Name="spellcheck" Range="[0,9)" Attributes={Fuzzy="gathering",Score="0.777777777777778",dynamic="",value="5"} NumericalAttributes={}/>,<Annotation ID="2003" Name="spellcheck" Range="[0,9)" Attributes={Fuzzy="katharine",Score="0.888888888888889",dynamic="",value="6"} NumericalAttributes={}/>,<Annotation ID="2004" Name="spellcheck" Range="[0,9)" Attributes={Fuzzy="katherine",Score="1",dynamic="",value="5"} NumericalAttributes={}/>,<Annotation ID="4001" Name="qsc_known_word" Range="[0,9)" Attributes={value="9"} NumericalAttributes={}/> 531f0d9d-6cb7-c0a8-27b6-5022c0ff2a0d

Most of the confusion comes from the now poorly named “EnablePhonetics” parameter, found in a Search web part’s DataProviderJSON configuration or in the Query Object Model.  The vestigial “EnablePhonetics” parameter enables Core Fuzzy Name Searching, which unfortunately has no phonetic matching features.  Name Suggestions is the nickname synonym service.  Names found in the nickname dictionary will have their nickname mappings added to the query. Use the following PowerShell to see what nicknames are handled by the service:

> Get-SPEnterpriseSearchLanguageResourcePhrase -type NickName -Language en-US -SearchApplication $ssa -Owner $owner | ft Phrase, Mapping

  Conclusion So what to do about the Cathrine Doe > Kathy Smith scenario?  First, decide on the purpose of your “People” search vertical.  Is it a people name directory look up?  An expertise search?  Should the sorting be influenced by social distance or your organizational chart?  Then pick your Ranking Model.  You might find that the “People Name search ranking model” better suits your needs than the default “People Search application ranking model.”  Take time to look at which Managed Properties each ranking model weighs most heavily.  You might even notice that the “People Search expertise ranking model” is very poorly named.]]> SharePoint 2013 Search: Ranked Result Blocks – The Definitive Answer https://docs.microsoft.com/archive/blogs/ericdixon/sharepoint-2013-search-ranked-result-blocks-the-definitive-answer Thu, 02 Apr 2015 10:03:00 GMT https://blogs.msdn.microsoft.com/ericdixon/2015/04/02/sharepoint-2013-search-ranked-result-blocks-the-definitive-answer/ I’ve been stumped for a long time on how Ranked Result Blocks really work.  When creating a Result Block, two options are presented: (1) This block is always shown above core results. (2) This block is ranked within core results (may not show).  The first option is self-explanatory.  The block will always show above the results, though it may display under another result block.  The second options says the block will be ranked with the other search results, but how is it ranked?

The statement “This block is ranked within core results” is misleading and has caused many SharePoint Search Admins to incorrectly believe that somehow the block has received a relevancy-ranking score and is blended in with the search result items returned by the original query. 

In my own personal experimentation, I noticed that if I created two result blocks, the last block created appeared in the first position and first block I created was in the fourth position.  How could something be “ranked within core results” if the positioning was absolute?

I searched online for some sort of documentation, but I found no answer.  I asked many of my search peers how result blocks were ranked; no one knew for sure; however, one lone opinion is that ranking is related to clicks. I finally found a lone reference in a Search Community forum supporting the assertion that the ranking is determined by clicks, so I tested the theory.  I created two result blocks and clicked like mad on the “lower ranked” block, but I couldn’t make it move and was unable to validate the theory.

The question arose again recently with one of my customers.  He had the same question: “Why do my Ranked Result Blocks always appear in the same positions?” I finally reached out to the Jedi Knight of all things search, Dan Pandre, and said, “Help! I’m trying to understand Ranked Result Blocks.”  Dan said, “Try not. Do, or do not. There is no try. There is only source code.”

The Definitive Answer

After digging through the code, with some guidance from Dan, I finally arrived at what I believe is the definitive answer.  Here are the facts:

  • Ranked Result Blocks are ranked according to a "predicted click %" score. 
  • A newly created Ranked Result Block has a “predicted click %” of 0.811.  This score will change over time according to the predictive click algorithm and the number of clicks within a Result Block (i.e., clicks on results in the block are registered with the block id).
  • Clicks within the Result Block are recorded and then compiled on a 15-minute interval by the Query Logging timer job.
  • When Result Blocks are triggered by a query and gathered for results, they are first sorted by greatest “predicted click %” score to smallest and then assigned to slots. A Ranked Result Block may appear in one of three slots (i.e., slots 0, 2 and 4) that are interleaved with the search results.  Only one Result Block per slot is allowed. 
  • Search results may appear in slots 1, 3 and 5.  Default settings place two search results, if available, in slots 1 and 3 and “the rest” in slot 5.  The effect is two results between each Result Block. 
  • Each slot has a minimum threshold that the Ranked Result Block “predictive click %” score must exceed.
  • To appear in Slot 0 (the top), a result block must have a “predictive click %” score greater than 0.5 (this is why a newly created block is almost always on top).   In order to appear in Slot 2, a Result block must have a “predictive click %” score greater than 0.18.  In order to appear in Slot 4, a Result Block must have a “predictive click %” score greater than 0.0.  Again, only one Result Block is allowed per slot.    

In summary, a Ranked Result Block is ranked by its “predictive click %” score relative to other Ranked Result Blocks scores.  It will appear in one of three predetermined slots if the “predictive click %” score exceeds the Slot threshold.

Below is an example of a search results page with Ranked Result Blocks in slots 0, 2 and 4 and search results in slots 1, 3 and 5.




 
 
 
 
 
 
 
 
 
 
 
 




]]> SharePoint 2013 Search: Managing Query Rules with PowerShell https://docs.microsoft.com/archive/blogs/ericdixon/sharepoint-2013-search-managing-query-rules-with-powershell Thu, 19 Mar 2015 16:21:00 GMT https://blogs.msdn.microsoft.com/ericdixon/2015/03/19/sharepoint-2013-search-managing-query-rules-with-powershell/ One thing I've noticed about Query Rules, Search Admins are really good at creating Query Rules... not so good at deleting them.  Query Rules become difficult to manage when the number of rules in the environment grows to several dozen, hundreds or even thousands.

As content and relevancy change with time, Query Rules may become invalid or unnecessary.  For example, a Promoted Result Query Rule (Best Bet) may now point to a resource that no longer exists or the document a Promoted Result points to which was previously buried deep in the search results is now the #1 result.

To this end I created a script that will help the Search Admin identify and delete "invalid" Query Rules.  The intention of the script is to identify three basic use cases, log and, if desired, delete the logged Query Rules.  The use cases are:

  1. The URL in a Promoted Result action of the Query Rule is no longer valid.
  2. The Promoted Result appears in the top search results.
  3. The Query Rule no longer has Actions.

The script may be downloaded from the TechNet Script Gallery.  Click on over to https://gallery.technet.microsoft.com/PowerShell-script-tests-b20b780d to learn more. 

]]>