how to use the sentence transformer embedding model instead of openai text-embedding-ada-002 model in azure openai

Question

how to use the sentence transformer embedding model instead of openai text-embedding-ada-002 model in azure openai

Amaaz Arshad 135

I want to use the sentence transformer embedding model instead of openai text-embedding-ada-002 model in azure openai.

In azure cognitive search, inside my index, I have already created the vectors using sentence-transformers/multi-qa-mpnet-base-dot-v1, now what I want to do is to use this model for querying also i.e. when using vector search on my data, but there is no deployment for this model in azure ai studio. How can I achieve this?

Saideep Anchuri 9,425 Microsoft External Staff Moderator

Hi Amaaz Arshad

sentence-transformers/multi-qa-mpnet-base-dot-v1 model is not there in Azure AI foundry yet, you can deploy a sentence-transformers-all-minilm-l6-v2 instead which is text-similarity model like previous one or leverage mlflow to log the desired model and deploy it as online endpoint - Reference on deploying mlflow model as online endpoint, Sentence transformer mlflow tutorial

If you have already been to able to generate vector embedding with concerned model. You can create a new column named "embeddings: in your panda frame containing data and search best match to your input query using cosine similarity.

client = AzureOpenAI(

  api_key = os.getenv("AZURE_OPENAI_API_KEY"),  

  api_version = "2024-02-01",

  azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT")

)
 
def generate_embeddings(text): # model = "deployment_name"

    # send text to your model endpoint in Azure machine learning or AI foundry to get back the embeddings          

df_bills['sentence_embd'] = df_bills["text"].apply(lambda x : generate_embeddings (x)) # updates a new columsn for embedding 
 
def cosine_similarity(a, b):

    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
 
def get_embedding(text)

              #generate a embedding again with your sentence transformer model endpoint         
 
def search_docs(df, user_query, top_n=4, to_print=True):

    embedding = get_embedding(

        user_query

    )

    df["similarities"] = df.ada_v2.apply(lambda x: cosine_similarity(x, embedding))
 
    res = (

        df.sort_values("similarities", ascending=False)

        .head(top_n)

    )

    if to_print:

        display(res)

    return res
 
 
res = search_docs(df_bills, "Can I get information on cable company tax revenue?", top_n=4)

Kindly refer below link: programming-language-python

Thank you.

Amaaz Arshad 135

Hi @Saideep Anchuri Thanks for your quick reply.

I was able to solve this by editing the embedding_dependency inside openai chat competition API. I deployed the sentence transformer embedding endpoint through Azure Functions and then utilized that endpoint inside the embedding_dependency parameter.

completion = client.chat.completions.create(  
    model=deployment,  
    messages=[
        {
            "role": "system",
            "content": "You are an AI assistant that helps people find information."
        }
    ],  
    past_messages=10,
    max_tokens=800,  
    temperature=0.7,  
    top_p=0.95,  
    frequency_penalty=0,  
    presence_penalty=0,  
    stop=None,  
    extra_body={  
        "data_sources": [  
            {  
                "type": "azure_search",  
                "parameters": {  
                    "endpoint": os.environ["AZURE_AI_SEARCH_ENDPOINT"],  
                    "index_name": os.environ["AZURE_AI_SEARCH_INDEX"],                     	
                    "authentication": {
                        "type": "api_key",
                        "key": "value"
                    },
                    "embedding_dependency": {
                        "type": "endpoint",
                        "endpoint": app_settings.azure_openai.embedding_endpoint,
                        "authentication": {
                            "type": "api_key",
                            "key": app_settings.azure_openai.embedding_key
                        }
                    }
                }  
            }  
        ]  
    }  
)

Accepted answer

0 additional answers

Your answer

Saideep Anchuri 9,425 Reputation points Microsoft External Staff Moderator

2025-03-19T10:35:50.4666667+00:00

Hi Amaaz Arshad

sentence-transformers/multi-qa-mpnet-base-dot-v1 model is not there in Azure AI foundry yet, you can deploy a sentence-transformers-all-minilm-l6-v2 instead which is text-similarity model like previous one or leverage mlflow to log the desired model and deploy it as online endpoint - Reference on deploying mlflow model as online endpoint, Sentence transformer mlflow tutorial

If you have already been to able to generate vector embedding with concerned model. You can create a new column named "embeddings: in your panda frame containing data and search best match to your input query using cosine similarity.

client = AzureOpenAI( api_key = os.getenv("AZURE_OPENAI_API_KEY"), api_version = "2024-02-01", azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT") ) def generate_embeddings(text): # model = "deployment_name" # send text to your model endpoint in Azure machine learning or AI foundry to get back the embeddings df_bills['sentence_embd'] = df_bills["text"].apply(lambda x : generate_embeddings (x)) # updates a new columsn for embedding def cosine_similarity(a, b): return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)) def get_embedding(text) #generate a embedding again with your sentence transformer model endpoint def search_docs(df, user_query, top_n=4, to_print=True): embedding = get_embedding( user_query ) df["similarities"] = df.ada_v2.apply(lambda x: cosine_similarity(x, embedding)) res = ( df.sort_values("similarities", ascending=False) .head(top_n) ) if to_print: display(res) return res res = search_docs(df_bills, "Can I get information on cable company tax revenue?", top_n=4)

Kindly refer below link: programming-language-python

Thank you.
Amaaz Arshad 135 Reputation points

2025-03-19T14:50:16.8733333+00:00

Hi @Saideep Anchuri Thanks for your quick reply.

I was able to solve this by editing the embedding_dependency inside openai chat competition API. I deployed the sentence transformer embedding endpoint through Azure Functions and then utilized that endpoint inside the embedding_dependency parameter.

completion = client.chat.completions.create( model=deployment, messages=[ { "role": "system", "content": "You are an AI assistant that helps people find information." } ], past_messages=10, max_tokens=800, temperature=0.7, top_p=0.95, frequency_penalty=0, presence_penalty=0, stop=None, extra_body={ "data_sources": [ { "type": "azure_search", "parameters": { "endpoint": os.environ["AZURE_AI_SEARCH_ENDPOINT"], "index_name": os.environ["AZURE_AI_SEARCH_INDEX"], "authentication": { "type": "api_key", "key": "value" }, "embedding_dependency": { "type": "endpoint", "endpoint": app_settings.azure_openai.embedding_endpoint, "authentication": { "type": "api_key", "key": app_settings.azure_openai.embedding_key } } } } ] } )

Answer 1

Hi Amaaz Arshad

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer.

Ask: how to use the sentence transformer embedding model instead of openai text-embedding-ada-002 model in azure openai

Solution: The issue is resolved. That you was able to solve this by editing the embedding_dependency inside openai chat competition API. I deployed the sentence transformer embedding endpoint through Azure Functions and then utilized that endpoint inside the embedding_dependency parameter.

completion = client.chat.completions.create(  
    model=deployment,  
    messages=[
        {
            "role": "system",
            "content": "You are an AI assistant that helps people find information."
        }
    ],  
    past_messages=10,
    max_tokens=800,  
    temperature=0.7,  
    top_p=0.95,  
    frequency_penalty=0,  
    presence_penalty=0,  
    stop=None,  
    extra_body={  
        "data_sources": [  
            {  
                "type": "azure_search",  
                "parameters": {  
                    "endpoint": os.environ["AZURE_AI_SEARCH_ENDPOINT"],  
                    "index_name": os.environ["AZURE_AI_SEARCH_INDEX"],                     	
                    "authentication": {
                        "type": "api_key",
                        "key": "value"
                    },
                    "embedding_dependency": {
                        "type": "endpoint",
                        "endpoint": app_settings.azure_openai.embedding_endpoint,
                        "authentication": {
                            "type": "api_key",
                            "key": app_settings.azure_openai.embedding_key
                        }
                    }
                }  
            }  
        ]  
    }  
)

If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.

Thank You.

Share via

how to use the sentence transformer embedding model instead of openai text-embedding-ada-002 model in azure openai

0 additional answers

Your answer