How to map custom field in sharepoint in indexer for azure cognitive search

Question

I am trying to update an indexer in cognitive search to map the custom field "Url" and "Doc Title" in sharepoint

with the index based on this article https://learn.microsoft.com/en-us/azure/search/search-howto-index-sharepoint-online. I also referenced this question https://learn.microsoft.com/en-us/answers/questions/659664/indexing-sharepoint-online-site-in-azure-search?orderby=oldest to map the custom fields.

I added the additional coulmns in datasource as mentioned in the article, updated the index and mapped the index with the custom fields in indexer.

However, when I use the cognitive search, the newly added fields "url" and "doc_title" are null in response while the other traditional fields(e.g metadata_spo_item_path, content) has content.

The additional columns in datasource container I added is as below.

"container" : { "name" : "useQuery", "query" : "includeLibrary=https://sanuker0.sharepoint.com/sites/OpenaiTeamsDemo/Sanuker%20doc/Forms/AllItems.aspx;additionalColumns=Url,DocTitle" }

The index I added in fields in index is as below.

"fields": [
....other indexes
{ "name": "url", "type": "Edm.String", "key": false, "searchable": true, "filterable": false, "sortable": false, "facetable": false },
{ "name": "doc_title", "type": "Edm.String", "key": false, "searchable": true, "filterable": false, "sortable": false, "facetable": false }
]

The fieldmapping I added in indexer is as below.

"fieldMappings" : [

    {

      "sourceFieldName" : "metadata_spo_site_library_item_id",

      "targetFieldName" : "id",

      "mappingFunction" : {

        "name" : "base64Encode"

      }

    },

    {

    "sourceFieldName": "Url",

    "targetFieldName": "url",

    "mappingFunction": null

    },

    {

    "sourceFieldName": "DocTitle",

    "targetFieldName": "doc_title",

    "mappingFunction": null

    }

]

I suspect

1. the column name in datasource is incorrect. How to retrieve the correct identification of the column name in SharePoint?
2. the field mapping in indexer is incorrect. The sourceFieldName should be the custom field name in SharePoint and the targetFieldName should be the index name. What should I revise on it?

Thanks.

Answer 1

Hi @Mary Ma,

I'm glad to hear you solve the problem ,if you have any issue about SharePoint, you are welcome to raise a ticket in this forum.

By the way, since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others." and according to the scenario introduced here: Answering your own questions on Microsoft Q&A, I would make a brief summary of this thread:

[How to map custom field in sharepoint in indexer for azure cognitive search]

Issue Symptom:

when using Cognitive Search, the newly added fields "url" and "doc_title" respond as null, while other traditional fields (eg metadata_spo_item_path, content) have content.

Solution:

Add a space in additional column in datasource. i.e. if the sharepoint column name "Doc Title", then have to add it as additionalColumns=Doc Title.

You could click the "Accept Answer" button for this summary to close this thread, and this can make it easier for other community member's to see the useful information when reading this thread. Thanks for your understanding!

---

Best Regards

Cheng Feng

Answer 2

@Mary Ma Apologies for late response and inconvenience on this, I had discussion with PG and confirmed that mapping custom fields through "Field Mapping" functionality is not supported in SharePoint preview connector.

It's suggested to create a custom skill https://learn.microsoft.com/en-us/azure/search/cognitive-search-custom-skill-web-api calling Microsoft Graph API to get additional details you would need: https://learn.microsoft.com/en-us/graph/use-the-api or creating a custom built-indexer with an Azure Function or Logic app with Microsoft Graph and Push API to get the details to the index (https://learn.microsoft.com/en-us/azure/search/search-what-is-data-import).

Having said that reached out to doc team for adding this limitation in document.

Hope this helps.

Please let us know if further query or issue remains. Thanks.

Answer 3

In the end, it is problem of spacing. If the column in sharepoint is "Doc Title", then add the addtionalColumn in datasource exactly as the column name i.e. additionalColumns=Doc Title