Hello Mads Helles,
Welcome to the Microsoft Q&A and thank you for posting your questions here.
Regarding your questions, I understand that you are facing three major challenges as listed below and also need some documentation or references. You would like to:
- Update Documents by adding, split or update documents
- Delete Documents.
- Refresh or Rebuild the Vector Index.
To answer your questions, I strongly believe you will be using Python and Azure AI Search, Azure Cognitive Search, custom vector index in a database, etc. the process may vary slightly but you can achieve the goal by following the below steps.
However, to update documents within your vector indexes:
- Get the modified .txt or .pdf files from your Azure Blob Storage.
- Use the relevant Azure SDK (like the Python SDK) or REST APIs to update the documents in your vector index.
- Call the necessary APIs or SDK methods to re-index the updated documents.
Secondly, to Splitting and Deleting Documents:
- If you need to split a large document into smaller ones, extract the relevant sections and create new, smaller documents.
- To delete documents, identify which ones you want to remove and use the appropriate APIs or SDK methods to delete them from the index.
Thirdly, to refresh or rebuild the index:
- Run your indexer on demand with a "reset" option. Resetting the indexer clears the high-water mark and performs a full reindex of all documents.
- Alternatively, schedule regular updates (hourly or daily) to keep your index current.
- Keep in mind that the indexer will stop executing when there are no more documents to load or refresh.
If you are using Azure Cognitive Search, you can trigger a reindexing process. This can be done by updating the indexer configuration to force a full reindex like this:
az search indexer reset --name <indexer-name> --index-name <index-name> --data-source-name <data-source-name>
Alternatively, you can use the Azure Portal:
Navigate to your Azure Cognitive Search service.
Go to the "Indexers" section.
Select your indexer and click "Run".
References
To read more and unlock step by steps documentation as requested:
Run or reset indexers - Azure AI Search
How to create vector indexes - Azure AI Studio
Incrementally Indexing documents with Azure AI Search
OpenAI API - Trying to create vectors and chunked data using Azure
Create or Update Index (Preview) - Azure AI Search.
Vector search - Azure AI Search.
Also:
Check the lists of documents and training available by the right side of this page: Additional resources.
Accept Answer
I hope this is helpful! Do not hesitate to let me know if you have any other questions.
** Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful ** so that others in the community facing similar issues can easily find the solution.
Best Regards,
Sina Salam