Share via

Teams Azure Open AI Embeddings And Redis Search On Uploaded File Using Bot

  • Code Sample
Browse code

This bot has been created using Bot Builder and Teams Toolkit, this Azure Open AI sample app demonstrates the user search functionality based on Azure Open AI embeddings, Redis DB search and completion APIs for user queries, and it is based on files uploaded by an admin using a bot.

Included Features

  • Bots
  • Adaptive Cards
  • Azure Open AI
  • Redis Search
  • Blob Storage

SolutionArchitecture

Interaction with bot - Admin

Admin-Functionality-GIF

Interaction with bot - User

User-Search-Functionality-GIF

Prerequisites

Run the app (Using Teams Toolkit for Visual Studio Code)

The simplest way to run this sample in Teams is to use Teams Toolkit for Visual Studio Code.

  1. Ensure you have downloaded and installed Visual Studio Code
  2. Install the Teams Toolkit extension
  3. Select File > Open Folder in VS Code and choose this samples directory from the repo
  4. Using the extension, sign in with your Microsoft 365 account where you have permissions to upload custom apps
  5. In the env/.env.local or env/.env.local.user file, fill all the required values for below and other values will be generated automatically once you debug/start the app.

SECRET_AZURE_OPENAPI_KEY=<Azure OpenAI Service Key>

REDIS_CONNECTION=redis://default:<REDIS_PASSWORD>=@<REDIS_HOST>:<REDIS_PORT>

EMBEDDING_MODEL_URL=https://<Azure OpenAI Service Name>.openai.azure.com/openai/deployments/<Your Deployed Embedding Model Name>/embeddings?api-version=2023-03-15-preview

COMPLETION_MODEL_URL=https://<Azure OpenAI Service Name>.openai.azure.com/openai/deployments/<Your Deployed Completion Model Name>

AZURE_STORAGE_CONNECTION_STRING='DefaultEndpointsProtocol=https;AccountName=<Storage Account Name>;AccountKey=<Your Account Key>;EndpointSuffix=core.windows.net'

Note: Update the Azure Open API version from 2023-03-15-preview to any specific or public version in all the files wherever applicable.

Note: If you are deploying the code, make sure that above mentioned values are properly updated at env/.env.dev or env/.env.dev.user wherever required.

  1. Select Debug > Start Debugging or F5 to run the app in a Teams web client.
  2. Alternatively use the Run and Debug Activity Panel in Visual Studio Code and click the Run and Debug green arrow button.
  3. In the browser that launches, select the Add button to install the app to Teams.

If you do not have permission to upload custom apps (sideloading), Teams Toolkit will recommend creating and using a Microsoft 365 Developer Program account - a free program to get your own dev environment sandbox that includes Teams

Setup

Note these instructions are for running the sample on your local machine, the tunnelling solution is required because the Teams service needs to call into the bot.

  1. Run ngrok - point to port 3978

    ngrok http 3978 --host-header="localhost:3978"

    Alternatively, you can also use the dev tunnels. Please follow Create and host a dev tunnel and host the tunnel with anonymous user access command as shown below:

    devtunnel host -p 3978 --allow-anonymous

  2. Setup for Bot

    In Azure portal, create a Azure Bot resource.

    • For bot handle, make up a name.
    • Select "Use existing app registration" (Create the app registration in Microsoft Entra ID beforehand.)
    • Choose "Accounts in any organizational directory (Any Azure AD directory - Multitenant)" in Authentication section in your App Registration to run this sample smoothly.
    • If you don't have an Azure account create an Azure free account here

    In the new Azure Bot resource in the Portal,

    • Ensure that you've enabled the Teams Channel
    • In Settings/Configuration/Messaging endpoint, enter the current https URL you were given by running the tunneling application. Append with the path /api/messages

  3. Clone the repository

    git clone https://github.com/OfficeDev/Microsoft-Teams-Samples.git

  4. In a terminal, navigate to samples/bot-ai-enterprise-search/nodejs

  5. Install modules

    npm install
  1. Update the env/.env.local or env/.env.local.user file, fill all the required values for below and other values will be generated automatically once you debug/start the app.

SECRET_AZURE_OPENAPI_KEY=<Azure OpenAI Service Key>

REDIS_CONNECTION=redis://default:<REDIS_PASSWORD>=@<REDIS_HOST>:<REDIS_PORT>

EMBEDDING_MODEL_URL=https://<Azure OpenAI Service Name>.openai.azure.com/openai/deployments/<Your Deployed Embedding Model Name>/embeddings?api-version=2023-03-15-preview

COMPLETION_MODEL_URL=https://<Azure OpenAI Service Name>.openai.azure.com/openai/deployments/<Your Deployed Completion Model Name>

AZURE_STORAGE_CONNECTION_STRING='DefaultEndpointsProtocol=https;AccountName=<Storage Account Name>;AccountKey=<Your Account Key>;EndpointSuffix=core.windows.net'

Note: Update the Azure Open API version from 2023-03-15-preview to any specific or public version in all the files whereever applicable.

  1. Run your bot at the command line:

    npm start

  2. This step is specific to Teams.

    • Edit the manifest.json contained in the appManifest folder (appManifest.admin and appManifest.user folders) to replace your Microsoft App Id (that was created when you registered your bot earlier) everywhere you see the place holder string <BOT_ID> (depending on the scenario the Microsoft App Id may occur multiple times in the manifest.json)

    • Also, update the <TEAMS_APP_ID> with unique Guid for both the manifest.json stored in (appManifest.admin and appManifest.user) as this app has 2 roles: Admin and User.

    • Edit the manifest.json for validDomains with base Url domain. E.g. if you are using ngrok it would be https://1234.ngrok-free.app then your domain-name will be 1234.ngrok-free.app and if you are using dev tunnels then your domain will be like: 12345.devtunnels.ms.

    • Zip up the contents of the appManifest folder (appManifest.admin and appManifest.user folders separately) to create a manifest.zip (Make sure that zip file does not contains any subfolder otherwise you will get error while uploading your .zip package)

    • Upload the manifest.zip to Teams (In Teams Apps/Manage your apps click "Upload an app". Browse to and Open the .zip file. At the next dialog, click the Add button.)

    • Add the app to personal scope (Supported app scope)

Running the sample

Admin Functionalities

  1. Adding the bot: add-App

  2. Welcome Card: welcome-card

  3. Admin can send a file to the bot as an attachment in the message compose section in Teams. This will be delivered to the bot as a Message Activity and the code in this sample fetches and saves the file in Azure blob and creates the embedding for uploaded file and save it in Redis Cache Database. file-upload-using-bot

  4. File Uploaded Successfully: uploaded-file-saved

  5. Now, admin can ask query based on the uploaded docs and Azure Open AI will generate the embedding for query and search the most relevant vectors for user query in the Redis search database based on the embeddings generated for uploaded file contents. ask-query

  6. Admin will get the final answer based on the uploaded docs using Azure Open AI completion model: final-answer

User Functionalities

  1. Adding the bot: add-App

  2. Welcome Card: welcome-card

  3. User can ask query based on the uploaded docs by admin and Azure Open AI will generate the embedding for query and search the most relevant vectors for user query in the Redis search database based on the embeddings generated for uploaded file contents. ask-query

  4. User will get the final answer based on the uploaded docs using Azure Open AI completion model: final-answer

Deploy to Azure

Deploy your project to Azure by following these steps:

From Visual Studio Code From TeamsFx CLI
  • Open Teams Toolkit, and sign into Azure by clicking the Sign in to Azure under the ACCOUNTS section from sidebar.
  • After you signed in, select a subscription under your account.
  • Open the Teams Toolkit and click Provision from DEPLOYMENT section or open the command palette and select: Teams: Provision.
  • Open the Teams Toolkit and click Deploy or open the command palette and select: Teams: Deploy.
  • Run command teamsfx account login azure.
  • Run command teamsfx provision --env dev.
  • Run command: teamsfx deploy --env dev.

Note: Provisioning and deployment may incur charges to your Azure Subscription.

Note: Once the provisioning and deployment steps are finished please update the manifest.json contained in the appManifest folders (appManifest.admin and appManifest.user folders) for validDomains with base Url domain. E.g. if your deployed web app service URL is: https://botaxxxxx.azurewebsites.net/ then your domain-name will be botaxxxxx.azurewebsites.net.

Also, make sure that below key/values are properly added to the configuration section of web app after code deployement.

"name": "AZURE_STORAGE_CONNECTION_STRING", "value": 'DefaultEndpointsProtocol=https;AccountName=<Storage Account Name>;AccountKey=<Your Account Key>;EndpointSuffix=core.windows.net'

"name": "BOT_ID", "value": "<BOT_ID>"

"name": "BOT_PASSWORD", "value": "<BOT_PASSWORD>"

"name": "COMPLETION_MODEL_URL", "value": "https://<Azure OpenAI Service Name>.openai.azure.com/openai/deployments/<Your Deployed Completion Model Name>"

"name": "EMBEDDING_MODEL_URL", "value": "https://<Azure OpenAI Service Name>.openai.azure.com/openai/deployments/<Your Deployed Embedding Model Name>/embeddings?api-version=2023-03-15-preview"

"name": "REDIS_CONNECTION", "value": redis://default:<REDIS_PASSWORD>=@<REDIS_HOST>:<REDIS_PORT>

"name": "RUNNING_ON_AZURE", "value": "1"

"name": "SECRET_AZURE_OPENAPI_KEY", "value": "<Your Azure Open API Key>"

"name": "WEBSITE_NODE_DEFAULT_VERSION", "value": "~18"

"name": "WEBSITE_RUN_FROM_PACKAGE", "value": "1"

Preview

Once the provisioning and deployment steps are finished, you can sideload your app.

Note: Please refer above Setup section for manifest configurations and sideload your packages (Admin/User) in Teams.

Further reading

Bot

AI, Redis And Blob Storage