What is machine learning?

Machine learning is a data science technique that allows computers to use existing data to forecast future behaviors, outcomes, and trends. By using machine learning, computers learn without being explicitly programmed. Machine learning tools use AI systems which provide the ability to identify patterns and create associations from experience with the data.

Automated machine learning forecasts or predictions can make applications and devices smarter. For example, when you shop online, machine learning helps recommend other products you might want based on what you've bought. Or when your credit card is swiped, machine learning compares the transaction to a database of transactions and helps detect fraud. And when your robot vacuum cleaner vacuums a room, machine learning helps it decide whether the job is done.

Machine learning tools to fit each task

Azure Machine Learning provides all the tools developers and data scientists need for their machine learning workflows, including:

You can even use MLflow to track metrics and deploy models or Kubeflow to build end-to-end workflow pipelines.

Build machine learning models in Python or R

Start training on your local machine using the Azure Machine Learning Python SDK or R SDK. Then, you can scale out to the cloud. With many available compute targets, like Azure Machine Learning compute and Azure Databricks, and with advanced hyperparameter tuning services, you can build better models faster by using the power of the cloud. You can also automate model training and tuning using the SDK.

Build machine learning models with no-code tools

For code-free or low-code training and deployment, try:

  • Azure Machine Learning designer (preview)

    Use the designer to prep data, train, test, deploy, manage, and track machine learning models without writing any code. There is no programming required, you visually connect datasets and modules to construct your model. Try out the designer tutorial.

    Learn more in the Azure Machine Learning designer overview article.

  • Automated machine learning (AutoML) UI

    Learn how to create AutoML experiments in the easy-to-use interface.

MLOps: Deploy and lifecycle management

Machine learning operations (MLOps) is based on DevOps principles and practices that increase the efficiency of workflows. For example, continuous integration, delivery, and deployment. MLOps applies these principles to the machine learning process, with the goal of:

  • Faster experimentation and development of models
  • Faster deployment of models into production
  • Quality assurance

When you have the right model, you can easily use it in a web service, on an IoT device, or from Power BI. For more information, see Deploy models with Azure Machine Learning.

Then you can manage your deployed models by using the Azure Machine Learning SDK for Python, Azure Machine Learning studio, or the Azure Machine Learning CLI.

These models can be consumed and return predictions either in real time or asynchronously on large quantities of data.

And with advanced machine learning pipelines, you can collaborate on each step from data preparation, model training and evaluation, through deployment. Pipelines allow you to:

  • Automate the end-to-end machine learning process in the cloud
  • Reuse components and only rerun steps when needed
  • Use different compute resources in each step
  • Run batch scoring tasks

If you want to use scripts to automate your machine learning workflow, the Azure Machine Learning CLI provides command-line tools that perform common tasks, such as submitting a training run or deploying a model.

To get started using Azure Machine Learning, see Next steps.

Automated machine learning

Data scientists spend an inordinate amount of time iterating over models during the experimentation phase. The whole process of trying out different algorithms and hyperparameter combinations until an acceptable model is built is extremely taxing for data scientists, due to the monotonous and non-challenging nature of work. While this is an exercise that yields massive gains in terms of the model efficacy, it sometimes costs too much in terms of time and resources and thus may have a negative return on investment (ROI).

This is where automated machine learning (AutoML) comes in. It uses the concepts from the research paper on probabilistic matrix factorization and implements an automated pipeline of trying out intelligently-selected algorithms and hypermeter settings, based on the heuristics of the data presented, keeping into consideration the given problem or scenario. The result of this pipeline is a set of models that are best suited for the given problem and dataset.

For more information on AutoML, see AutoML and MLOps with Azure Machine Learning.

Responsible ML

Throughout the development and use of AI systems, trust must be at the core. Trust in the platform, process, and models. As AI and autonomous systems integrate more into the fabric of society, it's important to proactively make an effort to anticipate and mitigate the unintended consequences of these technologies.

  • Understand your models and build for fairness: Explain model behavior and uncover features that have the most impact on predictions. Use built-in explainers for both glass-box and black-box models during model training and inference. Use interactive visualizations to compare models and perform what-if analysis to improve model accuracy. Test your models for fairness using state-of-the-art algorithms. Mitigate unfairness throughout the machine learning lifecycle, compare mitigated models, and make intentional fairness versus accuracy trade-offs as desired.
  • Protect data privacy and confidentiality: Build models that preserve privacy using the latest innovations in differential privacy, which injects precise levels of statistical noise in data to limit the disclosure of sensitive information. Identify data leaks and intelligently limit repeat queries to manage exposure risk. Use encryption and confidential machine learning (coming soon) techniques specifically designed for machine learning to securely build models using confidential data.
  • Control and govern through every step of the machine learning process: Access built-in capabilities to automatically track lineage and create an audit trial across the machine learning lifecycle. Obtain full visibility into the machine learning process by tracking datasets, models, experiments, code, and more. Use custom tags to implement model data sheets, document key model metadata, increase accountability, and ensure responsible process.

Learn more about how to implement Responsible ML.

Integration with other services

Azure Machine Learning works with other services on the Azure platform, and also integrates with open-source tools such as Git and MLflow.

Next steps