Auto Indexing Issue in Azure AI Foundry Playground

Question

Auto Indexing Issue in Azure AI Foundry Playground

Farah Diana Masri 40

index error

Hi, I have created one project to use gpt-4o with my own data. Currently, I want to use my own data with the model. I try to auto update my index daily, but I have issues after adding new data source and choose the Data in Azure AI Studio in the playground. The pane is keep loading for 1 day yesterday and not creating any new updated index. Please help me resolve this issue or is there any other way for me to auto updates my existing index? Thank you for your assistance.

romungi-MSFT 48,906 Reputation points Microsoft Employee Moderator

2024-12-03T05:25:12.1+00:00

@Farah Diana Masri I think you are trying to add your index from the chat playground screen from AI foundry portal. The playground from AI foundry is still in preview and there could be an issue with the screen while trying to add an index from this screen. I would recommend using the OpenAI studio chat playground and try to add the index from that screen.

Also AFAIK, the index refresh is not supported from AI foundry portal. You will have to refresh the index from AI search resource and then add then add it to your chat playground. I hope this helps!!
Farah Diana Masri 40 Reputation points

2024-12-03T07:38:32.5466667+00:00

Thanks for your reply, yes, I am using Azure AI Foundry. The purpose I am using Azure AI Foundry for auto indexing my existing index is because previously I have tried using this solution and it works, the index is updating for every 1 hour if I add any new files in my storage, and now I want to apply that solution back but since you mention Azure AI Foundry playground is still under preview, can you propose me any documentation or ways on how to updates my existing index? Below are the schedule updates for the index in Azure AI Foundry but I could not find it in Azure OpenAI Studio. Thank you.
Farah Diana Masri 40 Reputation points

2024-12-04T01:11:39.3933333+00:00

This is the screenshot of Azure AI Foundry, in + add a new data source pane at the playground. I have gone through the documentation you provide but I could not find the scheduling process. Hope you can give some insights, Thank you.
romungi-MSFT 48,906 Reputation points Microsoft Employee Moderator

2024-12-04T05:46:37.8833333+00:00

@Farah Diana Masri This is the documentation from AI search for scheduling on AI Search index.

https://learn.microsoft.com/en-us/azure/search/search-howto-schedule-indexers?tabs=portal

For the AI foundry scheduling screen, I am not seeing that in my case since I do not have an older search resource that supported this. Please try to schedule as mentioned in above link and checks if it works. Thanks!!
Farah Diana Masri 40 Reputation points

2024-12-05T00:29:52.43+00:00

Hi @romungi-MSFT , thank you. I have read the documentation and follow the steps to create schedule indexer, however, when I want to connect the data source in Azure AI Foundry playground with my latest created indexer, the loading state is too long and until right now I could not use that new index. Is that because the Add your data in Azure AI Foundry is still in preview that's why it is still not stable?
romungi-MSFT 48,906 Reputation points Microsoft Employee Moderator

2024-12-05T07:03:53.1933333+00:00

@Farah Diana Masri I do not see that issue with loading? I think something might be wrong with pulling your search resource connection. You might want to go back to your project settings or management center (button at bottom left in AI Foundry) and delete the search connection and re-add.

See the GIF below where my indexes are loading correctly.

Management center.

Find Connected resources tile and locate your search resource and delete it. You can re-add your search resource and then retry the Add your data. For ex:

I hope this helps!!

I will also convert my earlier comment on indexer schedule to answer since that functionality is part of search and not yet added to this portal for new resources. Thanks!!
Farah Diana Masri 40 Reputation points

2024-12-05T09:05:07.68+00:00

Hi @romungi-MSFT I think I am not explaining well. After I click on Create vector index after connecting with data source, then the loading consume a lot of times. The GIF you provide is steps before create the vector index.
romungi-MSFT 48,906 Reputation points Microsoft Employee Moderator

2024-12-05T13:02:55.3066667+00:00

@Farah Diana Masri I also did the create index and it works fine. Please see below:

I think you can retry readding the connection and check again along with enabling browser developer mode to understand which API call is failing. This can give an idea on what could be wrong with your resource.
romungi-MSFT 48,906 Reputation points Microsoft Employee Moderator

2024-12-06T10:15:51.4166667+00:00

@Farah Diana Masri I am curious to check if you are able to get this working? Thanks!!

1 answer

Your answer

romungi-MSFT 48,906 Reputation points Microsoft Employee Moderator

2024-12-03T05:25:12.1+00:00

@Farah Diana Masri I think you are trying to add your index from the chat playground screen from AI foundry portal. The playground from AI foundry is still in preview and there could be an issue with the screen while trying to add an index from this screen. I would recommend using the OpenAI studio chat playground and try to add the index from that screen.

Also AFAIK, the index refresh is not supported from AI foundry portal. You will have to refresh the index from AI search resource and then add then add it to your chat playground. I hope this helps!!
Farah Diana Masri 40 Reputation points

2024-12-03T07:38:32.5466667+00:00

Thanks for your reply, yes, I am using Azure AI Foundry. The purpose I am using Azure AI Foundry for auto indexing my existing index is because previously I have tried using this solution and it works, the index is updating for every 1 hour if I add any new files in my storage, and now I want to apply that solution back but since you mention Azure AI Foundry playground is still under preview, can you propose me any documentation or ways on how to updates my existing index? Below are the schedule updates for the index in Azure AI Foundry but I could not find it in Azure OpenAI Studio. Thank you.
Farah Diana Masri 40 Reputation points

2024-12-04T01:11:39.3933333+00:00

This is the screenshot of Azure AI Foundry, in + add a new data source pane at the playground. I have gone through the documentation you provide but I could not find the scheduling process. Hope you can give some insights, Thank you.
romungi-MSFT 48,906 Reputation points Microsoft Employee Moderator

2024-12-04T05:46:37.8833333+00:00

@Farah Diana Masri This is the documentation from AI search for scheduling on AI Search index.

https://learn.microsoft.com/en-us/azure/search/search-howto-schedule-indexers?tabs=portal

For the AI foundry scheduling screen, I am not seeing that in my case since I do not have an older search resource that supported this. Please try to schedule as mentioned in above link and checks if it works. Thanks!!
Farah Diana Masri 40 Reputation points

2024-12-05T00:29:52.43+00:00

Hi @romungi-MSFT , thank you. I have read the documentation and follow the steps to create schedule indexer, however, when I want to connect the data source in Azure AI Foundry playground with my latest created indexer, the loading state is too long and until right now I could not use that new index. Is that because the Add your data in Azure AI Foundry is still in preview that's why it is still not stable?
romungi-MSFT 48,906 Reputation points Microsoft Employee Moderator

2024-12-05T07:03:53.1933333+00:00

@Farah Diana Masri I do not see that issue with loading? I think something might be wrong with pulling your search resource connection. You might want to go back to your project settings or management center (button at bottom left in AI Foundry) and delete the search connection and re-add.

See the GIF below where my indexes are loading correctly.

Management center.

Find Connected resources tile and locate your search resource and delete it. You can re-add your search resource and then retry the Add your data. For ex:

I hope this helps!!

I will also convert my earlier comment on indexer schedule to answer since that functionality is part of search and not yet added to this portal for new resources. Thanks!!
Farah Diana Masri 40 Reputation points

2024-12-05T09:05:07.68+00:00

Hi @romungi-MSFT I think I am not explaining well. After I click on Create vector index after connecting with data source, then the loading consume a lot of times. The GIF you provide is steps before create the vector index.
romungi-MSFT 48,906 Reputation points Microsoft Employee Moderator

2024-12-05T13:02:55.3066667+00:00

@Farah Diana Masri I also did the create index and it works fine. Please see below:

I think you can retry readding the connection and check again along with enabling browser developer mode to understand which API call is failing. This can give an idea on what could be wrong with your resource.
romungi-MSFT 48,906 Reputation points Microsoft Employee Moderator

2024-12-06T10:15:51.4166667+00:00

@Farah Diana Masri I am curious to check if you are able to get this working? Thanks!!

Answer 1

romungi-MSFT 48,906 Microsoft Employee Moderator

@Farah Diana Masri Is this a screen shot of current AI foundry portal or a setting that was previously used while setting up? I think the scheduling process has changed since September 2024 as mentioned here. Since the ingestion APIs switched to integrated vectorization I think the same should be set on search when creating an index. See how to use integrated vectorization.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Share via

Auto Indexing Issue in Azure AI Foundry Playground

1 answer

Your answer