Add and configure models from Azure AI Foundry Models

You can decide and configure which models are available for inference in the Azure AI services resource model's inference endpoint. When a given model is configured, you can then generate predictions from it by indicating its model name or deployment name on your requests. No further changes are required in your code to use it.

In this article, you learn how to add a new model from Azure AI Foundry Models.

Prerequisites

To complete this article, you need:

• An Azure subscription. If you're using GitHub Models, you can upgrade your experience and create an Azure subscription in the process. Read Upgrade from GitHub Models to Azure AI Foundry Models if it's your case.
• An Azure AI services resource. For more information, see Create an Azure AI Foundry resource.

Add a model

As opposite to GitHub Models where all the models are already configured, the Azure AI Services resource allows you to control which models are available in your endpoint and under which configuration.

You can add all the models you need in the endpoint by using Azure AI Foundry for GitHub. In the following example, we add a Mistral-Large model in the service:

1. Go to Model catalog section in Azure AI Foundry for GitHub.

2. Scroll to the model you're interested in and select it.

   

3. You can review the details of the model in the model card.

4. Select Deploy.

5. For model providers that require more terms of contract, you are asked to accept those terms. This is the case for Mistral models for instance. Accept the terms on those cases by selecting Subscribe and deploy.

   

6. You can configure the deployment settings at this time. By default, the deployment receives the name of the model you're deploying. The deployment name is used in the model parameter for request to route to this particular model deployment. This allows you to also configure specific names for your models when you attach specific configurations. For instance o1-preview-safe for a model with a strict content filter. Use third-party models like Mistral, you can also configure the deployment to use a specific version of the model.

Tip

Each model can support different deployments types, providing different data residency or throughput guarantees. See deployment types for more details.

7. Use the Customize option if you need to change settings like content filter.

8. Select Deploy.

9. Once the deployment completes, the new model is listed in the page and it's ready to be used.

Use the model

Deployed models in Azure AI Foundry Models can be consumed using the Azure AI model's inference endpoint for the resource.

To use it:

1. Get the Azure AI model's inference endpoint URL and keys from the deployment page or the Overview page. If you're using Microsoft Entra ID authentication, you don't need a key.

2. When constructing your request, indicate the parameter model and insert the model deployment name you created.

   Python

   from azure.ai.inference.models import SystemMessage, UserMessage
   
   response = client.complete(
       messages=[
           SystemMessage(content="You are a helpful assistant."),
           UserMessage(content="Explain Riemann's conjecture in 1 paragraph"),
       ],
       model="mistral-large"
   )
   
   print(response.choices[0].message.content)
   

   JavaScript

   var messages = [
       { role: "system", content: "You are a helpful assistant" },
       { role: "user", content: "Explain Riemann's conjecture in 1 paragraph" },
   ];
   
   var response = await client.path("/chat/completions").post({
       body: {
           messages: messages,
           model: "mistral-large"
       }
   });
   
   console.log(response.body.choices[0].message.content)
   

   C#

   requestOptions = new ChatCompletionsOptions()
   {
       Messages = {
           new ChatRequestSystemMessage("You are a helpful assistant."),
           new ChatRequestUserMessage("Explain Riemann's conjecture in 1 paragraph")
       },
       Model = "mistral-large"
   };
   
   response = client.Complete(requestOptions);
   Console.WriteLine($"Response: {response.Value.Content}");
   

   Java

   List<ChatRequestMessage> chatMessages = new ArrayList<>();
   chatMessages.add(new ChatRequestSystemMessage("You are a helpful assistant"));
   chatMessages.add(new ChatRequestUserMessage("Explain Riemann's conjecture in 1 paragraph"));
   
   ChatCompletions chatCompletions = client.complete(new ChatCompletionsOptions(chatMessages));
   
   for (ChatChoice choice : chatCompletions.getChoices()) {
       ChatResponseMessage message = choice.getMessage();
       System.out.println("Response:" + message.getContent());
   }
   

   REST

   Request

   POST https://<resource>.services.ai.azure.com/models/chat/completions?api-version=2024-05-01-preview
   api-key: <api-key>
   Content-Type: application/json
   

   {
       "messages": [
           {
               "role": "system",
               "content": "You are a helpful assistant"
           },
           {
               "role": "user",
               "content": "Explain Riemann's conjecture in 1 paragraph"
           }
       ],
       "model": "mistral-large"
   }
   

3. When using the endpoint, you can change the model parameter to any available model deployment in your resource.

Additionally, Azure OpenAI models can be consumed using the Azure OpenAI in Azure AI Foundry Models endpoint in the resource. This endpoint is exclusive for each model deployment and has its own URL.

Model deployment customization

When creating model deployments, you can configure additional settings including content filtering and rate limits. Select the option Customize in the deployment wizard to configure it.

Note

Configurations may vary depending on the model you're deploying.

Related content

• Develop applications using Azure AI Foundry Models