Is there a workaround to index SharePoint site pages in Azure AI Search?

Question

Is there a workaround to index SharePoint site pages in Azure AI Search?

Irene Gunning 30

We are currently developing a chatbot using Azure Open AI, and we plan to use SharePoint as a data source. We have pages containing the data that we want the chatbot to reference. However, according to the documentation SharePoint Online indexer (preview) - Azure AI Search | Microsoft Learn, it states that indexing .aspx files is not supported and it only index documents in document libraries. Is there a workaround for this?

VenkateshDodda-MSFT 24,951 Reputation points Microsoft Employee Moderator

2024-11-05T06:59:43.64+00:00

@Irene Gunning Following up to see if you have a chance to check my previous response and helped, do let me know if you have any further questions on this.
Emily Du-MSFT 51,836 Reputation points Microsoft External Staff

2024-11-12T10:42:42.7866667+00:00

Hi Irene Gunning,

It is kindly following up to see if you have any remaining questions or concerns for this issue.

If VenkateshDodda's answer is helpful to this question, please remember to "Accept as answer" to close this case and this will help people in this forum who have similar questions to yours find the answer easier.

1 answer

Your answer

VenkateshDodda-MSFT 24,951 Reputation points Microsoft Employee Moderator

2024-11-05T06:59:43.64+00:00

@Irene Gunning Following up to see if you have a chance to check my previous response and helped, do let me know if you have any further questions on this.
Emily Du-MSFT 51,836 Reputation points Microsoft External Staff

2024-11-12T10:42:42.7866667+00:00

Hi Irene Gunning,

It is kindly following up to see if you have any remaining questions or concerns for this issue.

If VenkateshDodda's answer is helpful to this question, please remember to "Accept as answer" to close this case and this will help people in this forum who have similar questions to yours find the answer easier.

Answer 1

VenkateshDodda-MSFT 24,951 Microsoft Employee Moderator

@Irene Gunning Thanks for posting your question in Microsoft Q&A, apologize for any inconvenience caused on this.

As called out in the indexer documentation limitations, yes you cannot load .aspx file in SharePoint to an index.

One workaround to load .aspx to an indexer in SharePoint is by following the below steps:

Use the SharePoint APIs/SDKs to get the ASPX data or the specific method available that you choose to crawl through the data.
You can export the crawled data to blob storage and then you just need to use blob indexer for the content to be indexed accordingly (https://learn.microsoft.com/en-us/azure/search/search-howto-indexing-azure-blob-storage).

Hope this helps, let me know if you have any further questions on this.

Irene Gunning 30 Reputation points

2024-11-05T07:11:23.0366667+00:00

@VenkateshDodda-MSFT

Thank you for your response. So it means we need to develop a custom solution? Do you have any example projects or repo that shows how it is implemented through a custom solution?

Also the link you provided directs me to a "Bad Request" page:
Irene Gunning 30 Reputation points

2024-11-05T07:54:50.99+00:00

It's weird that my previous response is not posted. It looks like your site is bugged. I'll post it again:

Thank you for your response. So it means we need to develop a custom solution? Do you have any example projects or repo that shows how it is implemented through a custom solution? Also the link you provided directs me to a "Bad Request" page.

Edit: My previous comment showed up late, and I can't find a way to delete it. It looks like I need to post twice for my comments to appear. This also happened yesterday when my question didn’t show up right away so I thought it was not submitted but later appeared after I posted the same question thus the other question was a duplicate.
VenkateshDodda-MSFT 24,951 Reputation points Microsoft Employee Moderator

2024-11-05T13:03:27.5033333+00:00

@Irene Gunning Thanks for your follow questions on this.

Yes, you need to build a custom solution. Unfortunately, we don't have documentation or documentation steps for this.

Here is the link related to index blob storage which got broken in my previous message
https://learn.microsoft.com/en-us/azure/search/search-howto-indexing-azure-blob-storage

You can refer to this GitHub sample here which uses sharepointdataextractor class to extract the data from .docs and .pdf files and there is no specific callout that .aspx supports or not. If you face any issue, I would suggest you to report in that Github repo only.

Also note that this is beta stage not recommended for production.
Milan Gross 16 Reputation points

2024-11-07T03:38:53.27+00:00

This seems like a significant limitation considering the amount of content that is stored in SharePoint pages. Is there any roadmap for adding this to AI Search capabilities?
Irene Gunning 30 Reputation points

2024-11-12T12:38:09.6766667+00:00

Hi @VenkateshDodda-MSFT,I'm looking for a way to index Site Pages (.aspx files). These Site Pages are stored in the Site Pages library, not in document libraries, but I haven't found any instructions on how to do this. The first answer I received was ambiguous, so I followed up by asking for a sample project or repository. However, the sample repo provided was focused on extracting PDFs and DOCX files, not .aspx files. You mentioned, "Unfortunately, we don't have documentation or steps for this." Without proper guidance or documentation, it's unclear how a custom solution would work. Most of our clients are using SharePoint and we are looking for a way to use SharePoint as a data source of Azure AI Search for Azure Open AI. We are aware of the limitations and considerations posted in the documentation. However, we're specifically looking for workarounds that are not included in the documentation (https://learn.microsoft.com/en-us/azure/search/search-howto-index-sharepoint-online). Copilot Studio is a separate product with its own purchase requirements. Developing a custom solution is time-consuming, especially without examples, and can be expensive. Azure Blob Storage also incurs costs. We're seeking a simpler, more cost-effective alternative. Customers always prefer solutions that provide better value for their money.

Share via

Is there a workaround to index SharePoint site pages in Azure AI Search?

1 answer

Your answer