I have record in my vector index in azure cosmos db for mongodb vcore but connecting with azure open ai playground gives no results

Question

I have connected my vector Index in azure cosmosdb for mongodb vocre in the chat playground,
While asking question from my data it say sorry that retrieve documents do not have information for this although i have checked that the information is properly indexed and stored in my index , sometime it gives responses and sometime it dont why this is happening

although i have checked that this code is present in my vecctor index

Answer 1

Hello Ahmad Shafiq,

Thank you for posting your question in the Microsoft Q&A forum. 

The intermittent responses from your Azure Cosmos DB for MongoDB vCore vector index where queries sometimes return "no information" despite indexed data typically stem from query consistency issues or indexing latency. First, verify that the vector index is fully built and synchronized by checking the index status via db.collection.getIndexes(). If the index is still initializing, queries may miss recent updates. 

Next, review your query embedding logic: Ensure the vector search parameters (e.g., cosmosSearch command’s vectorEmbeddingPolicy) match the data format. Small deviations in embedding dimensions or similarity thresholds (k) can yield empty results. 

For reliability, enable diagnostic logs to capture failed queries and cross-check with Azure Monitor metrics for indexing delays. If using RAG (Retrieval-Augmented Generation), validate that the retrieved documents align with the LLM’s prompt context. 

Some reference documentation - https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/vcore/vector-search?tabs=diskann  

If the above answer helped, please do not forget to "Accept Answer" as this may help other community members to refer the info if facing a similar issue. Your contribution to the Microsoft Q&A community is highly appreciated. 

Answer 2

Hello Ahmad Shafiq,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that you are having inconsistent results when querying your vector index in Azure Cosmos DB for MongoDB vCore through the Azure OpenAI Playground, even though you've confirmed that the data is properly indexed.

You will need to check mismatches in how embeddings are generated and queried. One of the most common causes is using different embedding models or configurations for indexing and querying. For example, if you indexed your data using text-embedding-ada-002, but your queries are generated using a different model or with altered parameters, the similarity scores may drop below the retrieval threshold, resulting in no matches. You can learn more about embedding best practices here: Azure OpenAI Embeddings Guide.

Additionally, it's important to verify that the vector dimensions used during indexing match those used during querying. A mismatch in vector size like, 1536 vs. 1024 will silently fail to return results. You can inspect your index schema using the following MongoDB command:

db.collection.getIndexes()

This helps confirm that the vector index is correctly configured. More on this can be found in the Cosmos DB Vector Search documentation.

Another factor to consider is the indexing status and latency. If your index is still building or syncing, recent data might not be available for retrieval. Use Azure Monitor to check for indexing delays or replication lag. For guidance, refer to Monitor Cosmos DB Indexing.

To further troubleshoot, enable diagnostic logging to capture query vectors, similarity scores, and retrieved documents. This will help you determine whether the issue lies in the retrieval phase or in how the documents are being passed to the LLM. You can find troubleshooting steps here: Cosmos DB Troubleshooting Guide.

Finally, ensure that the Azure OpenAI Playground is correctly injecting the retrieved documents into the prompt. If the prompt is too long or improperly formatted, the model may ignore the context, leading to irrelevant or empty responses. Consider using a custom system message to guide the model’s behavior. More on prompt design is available here: Azure OpenAI Prompt Engineering.

If the issue persists, I recommend testing with a smaller dataset and a known query to isolate the problem. You may also consider rebuilding the index and validating each step with controlled inputs.

I hope this is helpful! Do not hesitate to let me know if you have any other questions or clarifications.

---

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.