Duplicate Q&A pairs created when updating documents in custom question answering KB.

Hi Team,

I'm using a Custom question answering to generate Q&A pairs based on uploaded documents. However, I'm experiencing an issue where when I update the content in the same document and re-upload it. it is creating duplicate Q&A pairs instead of updating the existing ones.

Can you please investigate and help me resolve this issue?

romungi-MSFT 42,206 Reputation points Microsoft Employee

2023-04-27T11:33:40.3666667+00:00

@REDDY, REDDYVALLA NAVEENKUMAR I believe you are using the language studio to update your document in the custom question answering project. In this case, updating the document with the same file name will actually add the entire content of the document to the knowledge base since the API basically imports all the content and extracts the same to create question & answer pairs. It will not check and replace the content as the document might have newer or updated questions which in some cases are required to maintain the KB.

I think there are two ways to avoid duplicate Q&A pairs, while uploading the content ensure you add the file with a different source name and file name. This helps you filter out the content by source name while editing in the studio and you can later delete the original source if the new content is correctly parsed and created.

Another way to avoid duplication is to select the Q&A pairs that require edit and export them from the KB and then modify the content as required and import the Q&A pairs back to the KB under the same source.

If you want to perform the same actions using API then Update Sources and Update Qnas API offer similar functionality.

I hope this helps!!
REDDY, REDDYVALLA NAVEENKUMAR 110 Reputation points

2023-04-28T06:37:20.2966667+00:00

Yes, that helps thanks for the Information @romungi-MSFT .

And also, we have SharePoint site contains content in Q&A pairs format and when we are adding that SharePoint URL as URL Source it says Unsupported/Invalid URL(s):

so that we can use smart URL refresh instead of uploading documents.
romungi-MSFT 42,206 Reputation points Microsoft Employee

2023-04-28T06:43:08.0166667+00:00
@REDDY, REDDYVALLA NAVEENKUMAR Yes, if you add a sharepoint link directly it will not be recognized. The guidelines available for QnAMaker while using sharepoint files should work for Custom Question answering too. To summarize these are pre-reqs that should work for Custom Q&A.

@REDDY, REDDYVALLA NAVEENKUMAR Yes, if you add a sharepoint link directly it will not be recognized. The guidelines available for QnAMaker while using sharepoint files should work for Custom Question answering too. To summarize these are pre-reqs that should work for Custom Q&A.

Cloud-based SharePoint - QnA Maker uses Microsoft Graph for permissions. If your SharePoint is on-premises, you won't be able to extract from SharePoint because Microsoft Graph won't be able to determine permissions.

URL format - QnA Maker only supports SharePoint urls which are generated for sharing and are of format https://\*.sharepoint.com

Thanks!!
REDDY, REDDYVALLA NAVEENKUMAR 110 Reputation points

2023-04-28T07:11:38.0166667+00:00

Yes @romungi-MSFT we are following guidelines provided.

Can we expect Microsoft to include this feature where SharePoint site URL like (https://*.sharepoint.com/sites/abc/SitePages/xyz.aspx) can be added without any alteration in near future, as it will definitely reduce the manual effort of generating Q&A pair directly from SharePoint content?

Thanks!!
KuppiReddygariSamthrushaReddy-3789 66 Reputation points

2023-05-23T08:16:29.2766667+00:00

@REDDY, REDDYVALLA NAVEENKUMAR Instead of adding share point url. you can try uploading it to blob and you can use that url in source.