Azure Cohere deployment does not work

Question

Azure Cohere deployment does not work

Eduard Nurmagomedov 0

Anyone managed to successfully deployed and use the Cohere models on azure? Here's a tutorial on how to use them, but I think it just does not work. I would be grateful for a functioning piece of code on how to consume them with the Cohere client.

https://github.com/Azure/azureml-examples/blob/main/sdk/python/foundation-models/cohere/rerank-cohere-client.ipynb

Anonymous

Hello Eduard Nurmagomedov,

As of March 31, 2025, the Cohere Re-rank v3 model for both English and multilingual applications has been officially deprecated. This means that it will no longer receive updates, support, or maintenance, which could affect its performance and compatibility with newer systems. For reference Deprecation for models in Azure AI model catalog.

To ensure seamless functionality and access to the latest improvements in model performance, it is recommended to transition to Cohere Re-rank v3.5. This updated version offers enhanced features, improved accuracy, and better support for diverse languages, making it a more robust choice for your re-ranking needs.

For your convenience, here is a sample code to guide you in deploying Cohere Re-rank v3.5.

import urllib.request
import json
 
# Request data goes here
# The example below assumes JSON formatting which may be updated
# depending on the format your endpoint expects.
# More information can be found here:
# https://docs.microsoft.com/azure/machine-learning/how-to-deploy-advanced-entry-script
data = {
        "query": "What is the capital of the United States?",
        "rank_fields": ["Title", "Content"],
        "documents": [
            {"Title": "Facts about Carson City", "Content": "Carson City is the capital city of the American state of Nevada. "},
            {"Title": "North Dakota", "Content" : "North Dakota is a state in the United States. 672,591 people lived in North Dakota in the year 2010. The capital and seat of government is Bismarck."},
            {"Title": "Micronesia", "Content" : "Micronesia, officially the Federated States of Micronesia, is an island nation in the Pacific Ocean, northeast of Papua New Guinea. The country is a sovereign state in free association with the United States. The capital city of Federated States of Micronesia is Palikir."}
        ],
        "top_n": 3
    }
 
body = str.encode(json.dumps(data))
 
url = 'https://Cohere-rerank-v3-5-yrbuo.eastus2.models.ai.azure.com/v1/rerank'
# Replace this with the primary/secondary key, AMLToken, or Microsoft Entra ID token for the endpoint
api_key = '<apikey>' #omit the api key
if not api_key:
    raise Exception("A key should be provided to invoke the endpoint")
 
 
headers = {'Content-Type':'application/json', 'Accept': 'application/json', 'Authorization':('Bearer '+ api_key)}
 
req = urllib.request.Request(url, body, headers)
 
try:
    response = urllib.request.urlopen(req)
 
    result = response.read()
    print(result)
except urllib.error.HTTPError as error:
    print("The request failed with status code: " + str(error.code))
 
    # Print the headers - they include the requert ID and the timestamp, which are useful for debugging the failure
    print(error.info())
    print(error.read().decode("utf8", 'ignore'))

Here is the output:

b'{"id":"fef8171e-abc0-4787-a64a-0510b6cb1d54","results":[{"index":1,"relevance_score":0.20813099},{"index":0,"relevance_score":0.1903328},{"index":2,"relevance_score":0.12261797}],"meta":{"api_version":{"version":"1"},"billed_units":{"search_units":1}}}'

Kindly refer this How to deploy Cohere Rerank models as serverless APIs - Azure Machine Learning.

I hope this helps. Do let me know if you have further queries.

Thank you!

Anonymous

2025-04-09T06:00:49.8533333+00:00

Hi @Eduard Nurmagomedov

Just checking back to see if the response was helpful.

Thank you.
Anonymous

2025-04-10T05:01:43.18+00:00

Hi @Eduard Nurmagomedov

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Eduard Nurmagomedov 0 Reputation points

2025-04-11T09:12:45.08+00:00

Hi,

Thanks for the response. This it helpful, and a similar request works for me now. However, I'm still having trouble because I would like to use the Cohere client, as in this example: https://github.com/Azure/azureml-examples/blob/main/sdk/python/foundation-models/cohere/rerank-cohere-client.ipynb

However, it does not work for me even if I use the 3.5 version. I get this error:

What do you think?
Anonymous

2025-04-15T05:47:01.1766667+00:00

Hi @Eduard Nurmagomedov

The client code available on GitHub is specifically designed to work with Cohere Reranked V3 models, which include both the English version and the multilingual version. This means the code uses APIs and features that are only compatible with these V3 models.

If you are using Reranked model version 3.5, you should refer to the updated sample code provided for that version.

I hope these helps you. Thank you!
Eduard Nurmagomedov 0 Reputation points

2025-04-15T09:07:57.1966667+00:00

Okay, but in your previous answer you write "As of March 31, 2025, the Cohere Re-rank v3 model for both English and multilingual applications has been officially deprecated."

So if the cohere.Client (or cohere.ClientV2) is designed to work only for v3, and v3 is officially deprecated, that means it's useless now? Would be nice to have some python client like that. I'm not confident in using the code in production otherwise.

Also, if v3 is deprecated, than the Github code that I sent is not functional anymore (as it's only for v3), so it would be useful if you removed it from official documentation (or updated it).
Anonymous

2025-04-16T01:10:55.2333333+00:00

Hi @Eduard Nurmagomedov

"The Cohere rank models have been deprecated. Our team is actively working on updates and will soon reflect these changes in the documentation.

Thank you!
Eduard Nurmagomedov 0 Reputation points

2025-04-16T07:39:04.8033333+00:00

Not a very helpful asnwer.
Anonymous

2025-04-16T12:05:05.9033333+00:00

Hi @Eduard Nurmagomedov

We apologize for any inconvenience caused. The Coherer Rank models have been deprecated, and our team is actively working on resolving this issue. Once we receive updates, the necessary changes will be reflected in the documentation and communicated to you promptly. We kindly ask for your patience during this time and appreciate your understanding. Rest assured, we are committed to providing you with the best possible service and will keep you informed as soon as the updates are available.

I hope these helps you. Thank you!