AI Foundry Deployment

Question

Hello,

Lets say that I have an agent that uses a fine-tuned LLM Model. How do I move this agent from dev environment to prod? I am using the Foundry interface to create the agent as well as fine tune the model. So I dont have any scripts that I can run with a different configurations.

Please help.

Regards,

Meena

Answer 1

Hello Meenakshi Sankar,

Welcome to Microsoft Q&A and Thanks for your detailed question.

Since you're using the Azure AI Foundry interface (UI-based workflow) to both fine-tune the model and create the agent and you don’t have scripts or IaC moving from Dev → Prod requires a structured recreation approach rather than a direct “promotion.”

Below is the complete and recommended process, merging best practices with Foundry-specific steps.

How to Move a Fine-Tuned Agent from Dev to Prod in AI Foundry

There is currently no one-click “Promote to Production” button in Foundry. You’ll need to move two components separately:

1. The fine-tuned model
2. The agent configuration

1: Create a Production Foundry Project

First, create a separate Production project inside Azure AI Foundry.

Recommended setup:

Separate Azure subscription (ideal for enterprise)

Separate Azure AI resource

Separate quota allocation

Proper RBAC controls

This ensures:

• Environment isolation
• Clean billing separation
• Enterprise-grade governance

2: Move / Recreate the Fine-Tuned Model

Fine-tuned models are scoped to the Azure AI resource where they were trained. They cannot be directly “moved” between resources.

Recommended Approach

Recreate the fine-tuning in Production:

Export or download:

Training dataset

  Validation dataset (if applicable)
  
     Hyperparameters used in Dev
     
        Base model version
        
        Switch to your **Production Foundry project**
        
        Re-run the fine-tuning job in Production
        
        Deploy the new fine-tuned model with a production deployment name (e.g., `model-prod`)
        

This ensures proper environment isolation and long-term maintainability.

Alternative

If Dev and Prod share:

Same subscription

Same Azure AI resource

You could Create a new deployment of the same fine-tuned model and Use a different deployment name for production

However, this is not recommended for SaaS or enterprise systems.

3: Recreate the Agent in Production

Agents are also project-scoped and cannot currently be exported/imported directly via UI.

You will need to manually replicate the configuration:

From Dev agent, document:

System instructions

Tool definitions

Function calling setup

Retrieval configuration (if using RAG)

Guardrails / safety settings

Model deployment name

Then:

1. Switch to the Production project
2. Create a new agent
3. Configure it exactly as in Dev
4. Point it to the new Production model deployment

4: Test in Production Environment

Before going live:

Run validation prompts

Verify tool integrations

Validate response formatting

Check performance and token usage

Confirm monitoring & logging are enabled

This ensures parity between Dev and Prod.

5: Deploy for Real-World Usage

Once validated:

• Update your application to point to the Production endpoint
• Monitor via Azure Monitor / Cost Management
• Confirm quota and scaling limits

Recommended Enterprise Setup

For production SaaS platforms, the ideal flow is:

Dev Subscription → Test Subscription → Prod Subscription

Each should have:

Separate Azure AI resources

Separate model deployments

Separate quota approvals

If you expect frequent Dev → Prod promotions:

Consider moving toward:

Infrastructure as Code (Bicep / ARM / Terraform)

SDK-based agent configuration instead of UI-only setup

CI/CD pipeline for model deployment

This allows:

• Version control
• Repeatable deployments
• Automated promotion workflows

I hope you find this helpful. If you have any further questions, please feel free to let us know.

Thank you!