Solution ideas
This article is a solution idea. If you'd like us to expand the content with more information, such as potential use cases, alternative services, implementation considerations, or pricing guidance, let us know by providing GitHub feedback.
This architecture shows you the components used to build, deploy, and manage high-quality models with Azure Machine Learning, a service for the end-to-end machine learning lifecycle.
Architecture
Download a Visio file of this architecture.
Note
The architecture described in this article is based on Azure Machine Learning's CLI and Python SDK v1. For more information on the new v2 SDK and CLI, see What is CLI and SDK v2.
Dataflow
- Bring together all your structured, unstructured, and semi-structured data (logs, files, and media) into Azure Data Lake Storage Gen2.
- Use Apache Spark in Azure Synapse Analytics to clean, transform, and analyze datasets.
- Build and train machine learning models in Azure Machine Learning.
- Control access and authentication for data and the machine learning workspace with Microsoft Entra ID and Azure Key Vault. Manage containers with Azure Container Registry.
- Deploy the machine learning model to a container using Azure Kubernetes Services, securing and managing the deployment with Azure VNets and Azure Load Balancer.
- Using log metrics and monitoring from Azure Monitor, evaluate model performance.
- Retrain models as necessary in Azure Machine Learning.
- Visualize data outputs with Power BI.
Components
- Azure Machine Learning is an enterprise-grade machine learning service for the end-to-end machine learning lifecycle.
- Azure Synapse Analytics is a unified service where you can ingest, explore, prepare, transform, manage, and serve data for immediate BI and machine learning needs.
- Azure Data Lake Storage Gen2 is a massively scalable and secure data lake for your high-performance analytics workloads.
- Azure Container Registry is a registry of Docker and Open Container Initiative (OCI) images, with support for all OCI artifacts. Build, store, secure, scan, replicate, and manage container images and artifacts with a fully managed, geo-replicated instance of OCI distribution.
- Azure Kubernetes Service Azure Kubernetes Service (AKS) offers serverless Kubernetes, an integrated continuous integration and continuous delivery (CI/CD) experience, and enterprise-grade security and governance. Deploy and manage containerized applications more easily with a fully managed Kubernetes service.
- Azure Monitor lets you collect, analyze, and act on telemetry data from your Azure and on-premises environments. Azure Monitor helps you maximize performance and availability of your applications and proactively identify problems in seconds.
- Azure Key Vault safeguards cryptographic keys and other secrets used by cloud apps and services.
- Azure Load Balancer load-balances internet and private network traffic with high performance and low latency. Load Balancer works across virtual machines, virtual machine scale sets, and IP addresses.
- Power BI is a suite of business analytics tools that deliver insights throughout your organization. Connect to hundreds of data sources, simplify data prep, and drive unplanned analysis. Produce beautiful reports, then publish them for your organization to consume on the web and across mobile devices.
Scenario details
Build, deploy, and manage high-quality models with Azure Machine Learning, a service for the end-to-end machine learning lifecycle. Use industry-leading MLOps (machine learning operations), open-source interoperability, and integrated tools on a secure, trusted platform designed for responsible machine learning.
Potential use cases
- Use machine learning as a service.
- Easy and flexible building interface.
- Wide range of supported algorithms.
- Easy implementation of web services.
- Great documentation for machine learning solutions.
Considerations
These considerations implement the pillars of the Azure Well-Architected Framework, which is a set of guiding tenets that can be used to improve the quality of a workload. For more information, see Microsoft Azure Well-Architected Framework.
Cost optimization
Cost optimization is about looking at ways to reduce unnecessary expenses and improve operational efficiencies. For more information, see Overview of the cost optimization pillar.
Contributors
This article is maintained by Microsoft. It was originally written by the following contributors.
Principal authors:
- Sheri Gilley | Senior Content Developer
- Larry Franks | Content Developer
- Lauryn Gayhardt | Content Developer
- Samantha Salgado | Content Developer
To see non-public LinkedIn profiles, sign in to LinkedIn.
Next steps
See documentation for the key services in this solution:
- Azure Machine Learning documentation
- Azure Synapse Analytics documentation
- Azure Data Lake Storage Gen2 documentation
- Azure Container Registry documentation
- Azure Kubernetes Service documentation
- Azure Monitor documentation
- Azure Key Vault documentation
- Azure Load Balancer documentation
- Power BI documentation
Related resources
See related guidance on the Azure Architecture Center:
- Compare the machine learning products and technologies from Microsoft
- Machine learning operations (MLOps) framework to upscale machine learning lifecycle with Azure Machine Learning
- Deploy AI and machine learning computing on-premises and to the edge
- Many models machine learning at scale with Azure Machine Learning
- Scale AI and machine learning initiatives in regulated industries
- Predict hospital readmissions by using traditional and automated machine learning techniques
- Secure research environment for regulated data