Can we use both unstructured and structured data together for creating chatbot using Azure openai services and cognitive search?

Question

I'm working on developing a chatbot using Azure services, specifically OpenAI and Cognitive Search. My goal is to combine both unstructured and structured data to create a more intelligent and context-aware chatbot.

What are the best practices for seamlessly integrating unstructured and structured data (from databases or other sources) in a chatbot?

What's the basic idea or best approach for chunking structured data before storing it in Azure Cognitive Search? Are there any specific considerations or strategies for optimal indexing and retrieval?

Can the same index in Azure Cognitive Search be used for both unstructured and structured data, or is it recommended to create separate indexes for each data type?

I would like to know:

Best practices for integrating unstructured and structured data in a chatbot. Any specific challenges or considerations when using Azure OpenAI services and Cognitive Search together. Examples or code snippets demonstrating the integration of these services for an effective chatbot. Any insights, advice, or resources would be greatly appreciated. Thanks in advance!

Answer 1

@Shamal Musthafa Yes, you can use both unstructured and structured data together for creating a chatbot using Azure OpenAI services and Cognitive Search. In fact, combining both types of data can help create a more intelligent and context-aware chatbot.

1. The best practice for integrating unstructured and structured data in a chatbot is to first define a clear schema for the structured data and then map the unstructured data to that schema. This will help ensure that the chatbot can easily access and use the data. When using Azure OpenAI services and Cognitive Search together, it's important to consider the specific requirements of each service and ensure that the data is formatted correctly for each.
2. When chunking structured data before storing it in Azure Cognitive Search, it's important to consider the size of the chunks and the specific requirements of the search queries. For optimal indexing and retrieval, it's recommended to chunk the data into smaller pieces that can be easily searched and retrieved. Additionally, it's important to ensure that the data is properly formatted and that any necessary metadata is included.
3. It is recommended to create separate indexes for each data type (unstructured and structured) in Azure Cognitive Search. This will help ensure that the data is properly organized and can be easily searched and retrieved.

Regarding the documentation and samples, I would suggest you visit the Azure Cognitive Search documentation. There we have a "Enrich with AI" section that goes further into Azure Cognitive Search + Azure OpenAI.