Azure ML: When should I use AutoML vs Model Catalog vs Notebooks?

Question

I'm learning Azure Machine Learning and I'm trying to understand the intended use case for the different ways of building ML solutions.

From what I understand:

• AutoML automates model training.

Model Catalog provides pre-trained foundation models.

Notebooks give full control over coding and experimentation.

However, I'm still unsure about the practical decision-making process.

Some questions I have:

What should be the priority when starting a new ML project?

How do you decide whether to use AutoML, a model from the Model Catalog, or build everything in a notebook?

What kinds of business problems are best suited for each approach?

Are there scenarios where one option should be avoided?

How do experienced Azure ML users typically make this decision in production projects?

I'd appreciate any real-world examples or decision frameworks.I'm learning Azure Machine Learning and I'm trying to understand the intended use case for the different ways of building ML solutions.

From what I understand:

AutoML automates model training.

Model Catalog provides pre-trained foundation models.

Notebooks give full control over coding and experimentation.

However, I'm still unsure about the practical decision-making process.

Some questions I have:

What should be the priority when starting a new ML project?

How do you decide whether to use AutoML, a model from the Model Catalog, or build everything in a notebook?

What kinds of business problems are best suited for each approach?

Are there scenarios where one option should be avoided?

How do experienced Azure ML users typically make this decision in production projects?

I'd appreciate any real-world examples or decision frameworks.

Answer 1

A practical way to choose is to start from the problem type and the level of customization required.

1. Use AutoML when the goal is to train and tune a model from data with less manual model selection.
   • Best suited for:
     • Classification: fraud detection, handwriting recognition, object detection
     • Regression: predicting numeric values such as automobile price
     • Forecasting: revenue, inventory, sales, customer demand
     • Computer vision: image classification, object detection, instance segmentation
     • NLP: text classification and named entity recognition
   • Strong fit when the priority is:
     • faster experimentation
     • less programming effort
     • applying built-in best practices
     • scaling into MLOps and pipelines later
   • Avoid it when the project requires highly custom modeling logic that must be coded manually.
2. Use the Model Catalog when the project needs a prebuilt foundation model for Generative AI scenarios.
   • Best suited for applications powered by large language models.
   • Strong fit when the priority is to discover and use existing models rather than train a traditional ML model from scratch.
   • Azure Machine Learning positions the model catalog as the hub for discovering and using a wide range of models for Generative AI applications.
   • Avoid it when the problem is a standard tabular prediction problem such as classification, regression, or forecasting from business data, where AutoML is a more direct fit.
3. Use Notebooks / SDK-based development when full control is required.
   • Best suited for:
     • custom experimentation
     • custom preprocessing and training code
     • authoring AutoML vision and NLP models through the Python SDK
     • inspecting outputs, metrics, and generated artifacts
   • Strong fit when experienced users need to customize, configure, and extend defaults.
   • AutoML itself can still be part of a notebook-driven workflow, since Azure ML supports SDK-based configuration and inspection of jobs and outputs.

A good starting priority for a new project is:

1. Define the business problem type
   • If it is classification, regression, forecasting, computer vision, or NLP model training from labeled data, start by evaluating AutoML.
   • If it is a Generative AI or LLM-based application, start by evaluating the Model Catalog.
   • If the solution needs custom code, custom experimentation, or nonstandard workflows, start in Notebooks.
2. Check how much control is needed
   • Low-code or faster iteration: AutoML
   • Pretrained foundation model reuse: Model Catalog
   • Maximum flexibility: Notebooks
3. Plan for production
   • Azure ML supports operationalizing at scale with MLOps and ML Pipelines.
   • For teams that need repeatable training, deployment, and lifecycle management, AutoML and SDK-based workflows both fit into production patterns.

A simple decision framework:

• Business data prediction from historical labeled data → AutoML
  • Example: predict sales demand, fraud risk, or numeric business outcomes.
• Generative AI application using existing foundation models → Model Catalog
  • Example: build an LLM-powered application.
• Custom research, advanced experimentation, or full-code workflow → Notebooks
  • Example: custom preprocessing, custom training logic, or deep inspection of metrics and artifacts.

Typical production-minded usage is often combined rather than exclusive:

• start with AutoML to establish a strong baseline quickly
• use Notebooks/SDK when customization or deeper control is needed
• use Model Catalog when the solution is based on foundation models instead of training a traditional model from scratch

Examples:

• Demand forecasting from OLTP history: AutoML forecasting
• Image defect detection with labeled images: AutoML computer vision
• Text classification from labeled documents: AutoML NLP
• LLM application prototype: Model Catalog, potentially with prompt flow in Azure Machine Learning
• Highly customized ML experiment: Notebooks

One caution from the documentation: AutoML test-data evaluation for the final recommended model is a preview capability.

---

References:

• What is automated machine learning (AutoML)?
• What is Azure Machine Learning?
• Getting started with AI experimentation
• Build an LLM solution in Azure - Microsoft Q&A