Deploy fine-tuned models in Microsoft Foundry

Deploy your fine-tuned model in the Microsoft Foundry portal with Serverless, Managed Compute, or Fireworks on Foundry. Start with a fine-tuned model that's ready for deployment.

For deployment comparisons, model compatibility, and artifact requirements, see the fine-tuning overview and its supported models table.

Prerequisites

• A fine-tuned model available in Foundry, with its model identifier and version.
• A Foundry project that supports the selected deployment option.
• The Foundry Owner role or a role with Microsoft.CognitiveServices/accounts/deployments/write on the target resource. Complete any additional provider setup listed in the example.
• Sufficient deployment quota or accelerator capacity in a supported region. Check model availability.

Deploy with Serverless

Deploy a compatible fine-tuned model with Serverless. Check the fine-tuning overview for the deployment types your model supports.

1. Open the Foundry portal and select your project.
2. Open the details view for your completed fine-tuning job, then select Deploy.
3. Choose Serverless.
4. Enter a name for your deployment.
5. Choose an available deployment type based on your requirements, and configure capacity within your available quota.
6. Create the deployment, then monitor its progress on the Models page.

When deployment completes, open the Foundry playground and test your model.

Deploy with Managed Compute (preview)

Important

Items marked preview in this article are currently in preview. This preview is provided without a service-level agreement, and Microsoft doesn't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

Deploy a compatible fine-tuned model with Managed Compute. For compatibility and deployment options, see the fine-tuning overview.

Before you begin

• Have a fine-tuned adapter that matches the base model and version.
• Have a base deployment that uses a LoRA-enabled deployment template. If you need to create one, follow Deploy open-source models with Managed Compute.
• Complete the Managed Compute permissions and quota setup.

Deploy in the portal

1. Open your project in the Foundry portal.
2. Open the details view for your completed fine-tuning job, then select Deploy.
3. Choose Managed Compute.
4. Enter a name for your deployment.
5. Select the compatible base deployment.
6. Review the configuration and select Deploy.
7. Wait for deployment to complete.

When deployment completes, open the Foundry playground and test your model.

Deploy with Fireworks on Foundry (preview)

Important

Items marked preview in this article are currently in preview. This preview is provided without a service-level agreement, and Microsoft doesn't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

Deploy a compatible fine-tuned model with Fireworks on Foundry. For compatibility and deployment options, see the fine-tuning overview.

Before you begin

• Have a fine-tuned model registered in Foundry and ready for Fireworks deployment.
• Ask a Subscription Owner or Subscription Contributor to enable Fireworks on Foundry.
• Complete the Fireworks permissions setup and confirm sufficient quota in a supported region.

Deploy in the portal

1. Open your project in the Foundry portal.
2. Open the details view for your completed fine-tuning job, then select Deploy.
3. Choose Fireworks.
4. Enter a name for your deployment.
5. Choose an available deployment type based on your requirements, and configure capacity within your available quota.
6. Review pricing and the provider terms, then select Deploy.
7. Wait for model preparation and deployment to complete.

When deployment completes, open the Foundry playground and test your model.

If your model isn't registered and you have local model files, follow Import custom models with Fireworks. Don't upload an already registered model again.

Use your deployed fine-tuned model

Test the deployment in the Foundry playground with a prompt representative of your task.

1. Open the Playground in your Foundry project.
2. Select the fine-tuned deployment you created, not the base-model deployment.
3. Send a test prompt and confirm that the model returns a response.

For repeatable evaluation, see Run evaluations from the Foundry portal or Cloud evaluation with the Microsoft Foundry SDK. To compare candidates, see View evaluation results.

Clean up your deployment

Remove the serving deployment when you no longer need it.

1. Open the Models page in the Foundry portal to find your deployment.
2. Select the deployment you created.
3. Select Delete and confirm the deletion.

Delete only the serving resources you created for this example.

Related content

• Fine-tuning overview.
• Run evaluations from the Foundry portal.
• View evaluation results.