Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
This article offers management recommendations for organizations running AI workloads on Azure. It focuses on Azure platform-as-a-service (PaaS) solutions for AI.
Manage AI deployments
Consistent deployment configurations enhance security, compliance, and operational efficiency across all AI environments. Organizations that standardize their deployment approach reduce configuration drift and ensure reliable performance. You must implement systematic deployment practices that align with your business requirements. Here's how:
Select the appropriate operating model for your organization. Deployment models create logical boundaries such as data domains or business functions to ensure autonomy, governance, and cost tracking. Deploy an instance of Azure AI Foundry for each business unit because sharing a single instance across multiple business units limits cost tracking and creates resource constraints. Define a project per use case and use hub-based projects only when teams require shared resources. For more information, see What type of Azure AI Foundry project do I need? and AI Foundry resource types.
Deploy to regions that meet your requirements. Model placement depends on specific latency, throughput, and compliance requirements that determine optimal performance. Check the Azure region product availability table to confirm support for required hardware, features, and data-residency rules before deployment to ensure performance and regulatory alignment.
Monitor AI deployment resources continuously. Resource monitoring captures performance data and identifies issues before they affect users. Diagnostic settings capture logs and metrics for all key services including Azure AI Foundry and Azure AI services. This monitoring provides visibility into system health and enables proactive issue resolution. See also Azure Monitor Baseline Alerts.
Manage deployment resources centrally. Centralized resource management provides consistent oversight and control across all AI deployments. Use the Management center in Azure AI Foundry to configure Foundry projects, track resource utilization, and govern access. This approach ensures standardized resource allocation and cost control. Also monitor costs in Azure AI Foundry.
Use Azure API Management as a unified gateway for multiple deployments. API Management provides consistent security, scalability, rate limiting, token quotas, and centralized monitoring when onboarding multiple applications or teams. This approach standardizes access patterns and reduces management overhead across your AI services. For more information, see Access Azure OpenAI and other language models through a gateway.
Manage AI models
Model monitoring ensures outputs align with Responsible AI principles and maintain accuracy over time. AI models experience drift due to changing data, user behaviors, or external factors that can lead to inaccurate results or ethical concerns. You must implement continuous monitoring to detect and address these changes proactively. Here's how:
Monitor model outputs for quality and alignment. Monitoring processes ensure workloads remain aligned with responsible AI targets and deliver expected results. Use Azure AI Foundry's observability features and monitor applications. For Azure AI Foundry Agent Service, monitor agent deployments.
Track model performance metrics continuously. Performance monitoring helps pinpoint issues when accuracy or response quality drops below acceptable thresholds. Monitor latency in response times and accuracy of vector search results through tracing in Azure AI Foundry.
Consider implementing a generative AI gateway for enhanced monitoring. Azure API Management enables logging and monitoring capabilities that platforms don't provide natively, including source IP collection, input text tracking, and output text analysis. This approach provides comprehensive audit trails and monitoring data. For more information, see Implement logging and monitoring for Azure OpenAI Service language models.
Choose compute. In Azure AI Foundry, compute resources support essential model deployments and fine-tuning. Standardize compute types, runtimes, and shutdown periods across compute instances, clusters, and serverless options.
Manage AI data
Data quality determines the accuracy and reliability of AI model outputs. Organizations that maintain high-quality data standards achieve better model performance and reduce the risk of biased or inaccurate results. You must implement systematic data management practices to ensure consistent model quality. Here's how:
Monitor data drift continuously. Data drift detection identifies when input data patterns change from training baselines, which can degrade model performance over time. Track accuracy and data drift in both generative and nongenerative AI workloads to ensure models remain relevant and responsive to current conditions. Use evaluations in Azure AI Foundry to establish monitoring baselines and detection thresholds.
Set up automated alerts for performance degradation. Alert systems provide early warning when model performance drops below acceptable thresholds, enabling proactive intervention before issues affect users. Configure custom alerts to detect performance deviations and trigger remediation workflows when models require retraining or adjustment.
Ensure quality data processing standards. Data preparation requirements differ between AI workload types but must maintain consistent quality standards across all implementations. For generative AI, structure grounding data in the correct format with appropriate chunking, enrichment, and embedding for optimal AI model consumption. For more information, see Guide to designing and developing a RAG solution.
Implement business continuity
Business continuity ensures AI services remain available during regional outages or service disruptions. Service interruptions can affect critical business operations that depend on AI capabilities, making continuity planning essential for organizational resilience. You must implement multi-region deployment strategies to maintain service availability. Here's how:
Deploy AI services across multiple regions. Multi-region deployments provide redundancy that maintains service availability when individual regions experience outages or capacity constraints. Implement multi-region deployment strategies for Azure AI Foundry and Azure OpenAI to ensure consistent service delivery.
Configure automated failover mechanisms. Automated failover reduces recovery time and ensures continuous service delivery when primary regions become unavailable. Set up traffic routing and load balancing between regions to enable seamless transitions during service disruptions.