Events
Mar 17, 11 PM - Mar 21, 11 PM
Join the meetup series to build scalable AI solutions based on real-world use cases with fellow developers and experts.
Register nowThis browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
This article shows you how to deploy and run the Chat with your data sample for Java. This sample implements a chat app using Java, Azure OpenAI Service, and Retrieval Augmented Generation (RAG) in Azure AI Search to get answers about employee benefits at a fictitious company. The app is seeded with PDF files including the employee handbook, a benefits document, and a list of company roles and expectations.
In this article, you accomplish the following tasks:
Once you complete this article, you can start modifying the new project with your custom code.
This article is part of a collection of articles that show you how to build a chat app using Azure OpenAI Service and Azure AI Search. Other articles in the collection include:
Note
This article uses one or more AI app templates as the basis for the examples and guidance in the article. AI app templates provide you with well-maintained, easy to deploy reference implementations that help to ensure a high-quality starting point for your AI apps.
A simple architecture of the chat app is shown in the following diagram:
Key components of the architecture include:
Most resources in this architecture use a basic or consumption pricing tier. Consumption pricing is based on usage, which means you only pay for what you use. To complete this article, there's a charge but it's minimal. When you're done with the article, you can delete the resources to stop incurring charges.
Learn more about cost in the sample repo.
A development container environment is available with all dependencies required to complete this article. You can run the development container in GitHub Codespaces (in a browser) or locally using Visual Studio Code.
To use this article, you need the following prerequisites:
Microsoft.Authorization/roleAssignments/write
permissions, such as User Access Administrator or Owner.Begin now with a development environment that has all the dependencies installed to complete this article.
GitHub Codespaces runs a development container managed by GitHub with Visual Studio Code for the Web as the user interface. For the most straightforward development environment, use GitHub Codespaces so that you have the correct developer tools and dependencies preinstalled to complete this article.
Important
All GitHub accounts can use Codespaces for up to 60 hours free each month with two core instances. For more information, see GitHub Codespaces monthly included storage and core hours.
Right-click on the following button, and select Open link in new windows in order to have both the development environment and the documentation available at the same time.
On the Create codespace page, review the codespace configuration settings, and then select Create Codespace.
Wait for the codespace to start. This startup process can take a few minutes.
In the terminal at the bottom of the screen, sign in to Azure with the Azure Developer CLI.
azd auth login --use-device-code
Copy the code from the terminal and then paste it into a browser. Follow the instructions to authenticate with your Azure account.
The remaining tasks in this article take place in the context of this development container.
The sample repository contains all the code and configuration files you need to deploy a chat app to Azure. The following steps walk you through the process of deploying the sample to Azure.
Important
Azure resources created in this section incur immediate costs, primarily from the Azure AI Search resource. These resources might accrue costs even if you interrupt the command before it fully executes.
Provision the Azure resources and deploy the source code by running the following command:
azd up
If you're prompted to enter an environment name, keep it short and lowercase, for example, myenv
. It's used as part of the resource group name.
When prompted, select a subscription to create the resources in.
When you're prompted to select a location the first time, select a location near you. This location is used for most the resources including hosting.
If you're prompted for a location for the OpenAI model, select a location that is near you. If the same location is available as your first location, select that.
Wait until app is deployed, which can take 5-10 minutes to complete.
After the application successfully deploys, you see a URL displayed in the terminal.
Select that URL labeled Deploying service web
to open the chat application in a browser.
The chat app is preloaded with employee benefits information from PDF files. You can use the chat app to ask questions about the benefits. The following steps walk you through the process of using the chat app.
In the browser, select or enter What is included in my Northwind Health Plus plan that is not in standard? in the chat text box.
From the answer, select one of the citations.
In the right-pane, use the tabs to understand how the answer was generated.
Tab | Description |
---|---|
Thought process | Script of the interactions in chat. |
Supporting content | Includes the information to answer your question and the source material. |
Citation | Displays the PDF page that contains the citation. |
When you're done, select the selected tab again to close the pane.
The OpenAI model and the settings that are used to interact with the model determine the intelligence of the chat app.
Setting | Description |
---|---|
Override prompt template | Prompt that is used to generate the answer. |
Retrieve this many search results | Number of search results that are used to generate the answer. You can see these sources returned in the Thought process and Supporting content tabs of the citation. |
Exclude category | Category of documents that are excluded from the search results. |
Use semantic ranker for retrieval | Feature of Azure AI Search that uses machine learning to improve the relevance of search results. |
Use query-contextual summaries instead of whole documents | When both Use semantic ranker and Use query-contextual summaries are checked, the LLM uses captions extracted from key passages, instead of all the passages, in the highest ranked documents. |
Suggest follow-up questions | Have the chat app suggest follow-up questions based on the answer. |
Retrieval mode | Vectors + Text means that the search results are based on the text of the documents and the embeddings of the documents. Vectors means that the search results are based on the embeddings of the documents. Text means that the search results are based on the text of the documents. |
Stream chat completion responses | Stream response instead of waiting until the complete answer is available for a response. |
The following steps walk you through the process of changing the settings.
In the browser, select the Developer Settings tab.
Select the Suggest follow-up questions checkbox and ask the same question again.
What is my deductible?
The chat returns suggested follow-up questions such as these:
1. What is the cost sharing for out-of-network services?
2. Are preventive care services subject to the deductible?
3. How does the prescription drug deductible work?
In the Settings tab, deselect Use semantic ranker for retrieval.
Ask the same question again?
What is my deductible?
What is the difference in the answers?
For example the response, which used the Semantic ranker provided a single answer: The deductible for the Northwind Health Plus plan is $2,000 per year
.
The response without semantic ranking returned an answer, which required more work to get the answer: Based on the information provided, it is unclear what your specific deductible is. The Northwind Health Plus plan has different deductible amounts for in-network and out-of-network services, and there is also a separate prescription drug deductible. I would recommend checking with your provider or referring to the specific benefits details for your plan to determine your deductible amount
.
The Azure resources created in this article are billed to your Azure subscription. If you don't expect to need these resources in the future, delete them to avoid incurring more charges. Use the following command to delete the Azure resources and remove the source code:
azd down --purge
Deleting the GitHub Codespaces environment ensures that you can maximize the amount of free per-core hours entitlement you get for your account.
Important
For more information about your GitHub account's entitlements, see GitHub Codespaces monthly included storage and core hours.
Sign into the GitHub Codespaces dashboard.
Locate your currently running Codespaces sourced from the Azure-Samples/azure-search-openai-demo-java GitHub repository.
Open the context menu for the codespace and then select Delete.
The app is separated out into two apps:
The backend /chat
API steps through the process of getting the answer:
This sample repository offers troubleshooting information. If your issue isn't addressed, log it in the repository's Issues.
Events
Mar 17, 11 PM - Mar 21, 11 PM
Join the meetup series to build scalable AI solutions based on real-world use cases with fellow developers and experts.
Register nowTraining
Module
Implement Retrieval Augmented Generation (RAG) with Azure OpenAI Service - Training
Learn how to use your own data with the Azure OpenAI Service.
Certification
Microsoft Certified: Azure AI Engineer Associate - Certifications
Design og implementer en Azure AI-løsning ved hjælp af Azure AI-tjenester, Azure AI Search og Azure Open AI.
Documentation
Get started with the chat using your own data sample for Python - Python on Azure
Get started with Python and search across your own data by using a chat app sample implemented using Azure OpenAI Service and Retrieval Augmented Generation (RAG) in Azure AI Search. Easily deploy with Azure Developer CLI. This article uses the Azure AI Reference Template sample.
Develop Java Apps that Use Azure AI Services - Java on Azure
This article provides an organized list of resources about Azure AI scenarios for Java developers, including documentation and code samples.
Azure AI for Java documentation
Get started using Azure AI with Java.