Tutorial: Model development on a cloud workstation

Learn how to develop a training script with a notebook on an Azure Machine Learning cloud workstation. This tutorial covers the basics you need to get started:

• Set up and configuring the cloud workstation. Your cloud workstation is powered by an Azure Machine Learning compute instance, which is pre-configured with environments to support your various model development needs.
• Use cloud-based development environments.
• Use MLflow to track your model metrics, all from within a notebook.

Prerequisites

To use Azure Machine Learning, you'll first need a workspace. If you don't have one, complete Create resources you need to get started to create a workspace and learn more about using it.

Start with Notebooks

The Notebooks section in your workspace is a good place to start learning about Azure Machine Learning and its capabilities. Here you can connect to compute resources, work with a terminal, and edit and run Jupyter Notebooks and scripts.

1. Sign in to Azure Machine Learning studio.

2. Select your workspace if it isn't already open.

3. On the left navigation, select Notebooks.

4. If you don't have a compute instance, you'll see Create compute in the middle of the screen. Select Create compute and fill out the form. You can use all the defaults. (If you already have a compute instance, you'll instead see Terminal in that spot. You'll use Terminal later in this tutorial.)

   

Set up a new environment for prototyping (optional)

In order for your script to run, you need to be working in an environment configured with the dependencies and libraries the code expects. This section helps you create an environment tailored to your code. To create the new Jupyter kernel your notebook connects to, you'll use a YAML file that defines the dependencies.

• Upload a file.

  Files you upload are stored in an Azure file share, and these files are mounted to each compute instance and shared within the workspace.

  1. Download this conda environment file, workstation_env.yml to your computer by using the Download raw file button at the top right.

  1. Select Add files, then select Upload files to upload it to your workspace.

     

  2. Select Browse and select file(s).

  3. Select workstation_env.yml file you downloaded.

  4. Select Upload.

  You'll see the workstation_env.yml file under your username folder in the Files tab. Select this file to preview it, and see what dependencies it specifies. You'll see contents like this:

  name: workstation_env
  # This file serves as an example - you can update packages or versions to fit your use case
  dependencies:
    - python=3.8
    - pip=21.2.4
    - scikit-learn=0.24.2
    - scipy=1.7.1
    - pandas>=1.1,<1.2
    - pip:
      - mlflow-skinny 
      - azureml-mlflow
      - psutil>=5.8,<5.9
      - ipykernel~=6.0
      - matplotlib
  

• Create a kernel.

  Now use the Azure Machine Learning terminal to create a new Jupyter kernel, based on the workstation_env.yml file.

  1. Select Terminal to open a terminal window. You can also open the terminal from the left command bar:

     

  2. If the compute instance is stopped, select Start compute and wait until it's running.

     

  3. Once the compute is running, you see a welcome message in the terminal, and you can start typing commands.

  4. View your current conda environments. The active environment is marked with a *.

     conda env list
     

  5. If you created a subfolder for this tutorial, cd to that folder now.

  6. Create the environment based on the conda file provided. It takes a few minutes to build this environment.

     conda env create -f workstation_env.yml
     
     

  7. Activate the new environment.

      conda activate workstation_env
     

  8. Validate the correct environment is active, again looking for the environment marked with a *.

     conda env list
     

  9. Create a new Jupyter kernel based on your active environment.

     python -m ipykernel install --user --name workstation_env --display-name "Tutorial Workstation Env" 
     

  10. Close the terminal window.

You now have a new kernel. Next you'll open a notebook and use this kernel.

Create a notebook

1. Select Add files, and choose Create new file.

   

2. Name your new notebook develop-tutorial.ipynb (or enter your preferred name).

3. If the compute instance is stopped, select Start compute and wait until it's running.

   

4. You'll see the notebook is connected to the default kernel in the top right. Switch to use the Tutorial Workstation Env kernel if you created the kernel.

Develop a training script

In this section, you develop a Python training script that predicts credit card default payments, using the prepared test and training datasets from the UCI dataset.

This code uses sklearn for training and MLflow for logging the metrics.

1. Start with code that imports the packages and libraries you'll use in the training script.

   import os
   import argparse
   import pandas as pd
   import mlflow
   import mlflow.sklearn
   from sklearn.ensemble import GradientBoostingClassifier
   from sklearn.metrics import classification_report
   from sklearn.model_selection import train_test_split

2. Next, load and process the data for this experiment. In this tutorial, you read the data from a file on the internet.

   # load the data
   credit_df = pd.read_csv(
       "https://azuremlexamples.blob.core.windows.net/datasets/credit_card/default_of_credit_card_clients.csv",
       header=1,
       index_col=0,
   )
   
   train_df, test_df = train_test_split(
       credit_df,
       test_size=0.25,
   )

3. Get the data ready for training:

   # Extracting the label column
   y_train = train_df.pop("default payment next month")
   
   # convert the dataframe values to array
   X_train = train_df.values
   
   # Extracting the label column
   y_test = test_df.pop("default payment next month")
   
   # convert the dataframe values to array
   X_test = test_df.values

4. Add code to start autologging with MLflow, so that you can track the metrics and results. With the iterative nature of model development, MLflow helps you log model parameters and results. Refer back to those runs to compare and understand how your model performs. The logs also provide context for when you're ready to move from the development phase to the training phase of your workflows within Azure Machine Learning.

   # set name for logging
   mlflow.set_experiment("Develop on cloud tutorial")
   # enable autologging with MLflow
   mlflow.sklearn.autolog()

5. Train a model.

   # Train Gradient Boosting Classifier
   print(f"Training with data of shape {X_train.shape}")
   
   mlflow.start_run()
   clf = GradientBoostingClassifier(n_estimators=100, learning_rate=0.1)
   clf.fit(X_train, y_train)
   
   y_pred = clf.predict(X_test)
   
   print(classification_report(y_test, y_pred))
   # Stop logging for this model
   mlflow.end_run()

   Note

   You can ignore the mlflow warnings. You'll still get all the results you need tracked.

Iterate

Now that you have model results, you may want to change something and try again. For example, try a different classifier technique:

# Train  AdaBoost Classifier
from sklearn.ensemble import AdaBoostClassifier

print(f"Training with data of shape {X_train.shape}")

mlflow.start_run()
ada = AdaBoostClassifier()

ada.fit(X_train, y_train)

y_pred = ada.predict(X_test)

print(classification_report(y_test, y_pred))
# Stop logging for this model
mlflow.end_run()

Note

You can ignore the mlflow warnings. You'll still get all the results you need tracked.

Examine results

Now that you've tried two different models, use the results tracked by MLFfow to decide which model is better. You can reference metrics like accuracy, or other indicators that matter most for your scenarios. You can dive into these results in more detail by looking at the jobs created by MLflow.

1. On the left navigation, select Jobs.

   

2. Select the link for Develop on cloud tutorial.

3. There are two different jobs shown, one for each of the models you tried. These names are autogenerated. As you hover over a name, use the pencil tool next to the name if you want to rename it.

4. Select the link for the first job. The name appears at the top. You can also rename it here with the pencil tool.

5. The page shows details of the job, such as properties, outputs, tags, and parameters. Under Tags, you'll see the estimator_name, which describes the type of model.

6. Select the Metrics tab to view the metrics that were logged by MLflow. (Expect your results to differ, as you have a different training set.)

   

7. Select the Images tab to view the images generated by MLflow.

   

8. Go back and review the metrics and images for the other model.

Create a Python script

Now create a Python script from your notebook for model training.

1. On the notebook toolbar, select the menu.

2. Select Export as> Python.

   

3. Name the file train.py.

4. Look through this file and delete the code you don't want in the training script. For example, keep the code for the model you wish to use, and delete code for the model you don't want.

   • Make sure you keep the code that starts autologging (mlflow.sklearn.autolog()).
   • You may wish to delete the autogenerated comments and add in more of your own comments.
   • When you run the Python script interactively (in a terminal or notebook), you can keep the line that defines the experiment name (mlflow.set_experiment("Develop on cloud tutorial")). Or even give it a different name to see it as a different entry in the Jobs section. But when you prepare the script for a training job, that line won't work and should be omitted - the job definition includes the experiment name.
   • When you train a single model, the lines to start and end a run (mlflow.start_run() and mlflow.end_run()) are also not necessary (they'll have no effect), but can be left in if you wish.

5. When you're finished with your edits, save the file.

You now have a Python script to use for training your preferred model.

Run the Python script

For now, you're running this code on your compute instance, which is your Azure Machine Learning development environment. Tutorial: Train a model shows you how to run a training script in a more scalable way on more powerful compute resources.

1. On the left, select Open terminal to open a terminal window.

   

2. View your current conda environments. The active environment is marked with a *.

   conda env list
   

3. If you created a new kernel, activate it now:

   conda activate workstation_env
   

4. If you created a subfolder for this tutorial, cd to that folder now.

5. Run your training script.

   python train.py
   

Note

You can ignore the mlflow warnings. You'll still get all the metric and images from autologging.

Examine script results

Go back to Jobs to see the results of your training script. Keep in mind that the training data changes with each split, so the results differ between runs as well.

Clean up resources

If you plan to continue now to other tutorials, skip to Next steps.

Stop compute instance

If you're not going to use it now, stop the compute instance:

1. In the studio, in the left navigation area, select Compute.
2. In the top tabs, select Compute instances
3. Select the compute instance in the list.
4. On the top toolbar, select Stop.

Delete all resources

Important

The resources that you created can be used as prerequisites to other Azure Machine Learning tutorials and how-to articles.

If you don't plan to use any of the resources that you created, delete them so you don't incur any charges:

1. In the Azure portal, select Resource groups on the far left.

2. From the list, select the resource group that you created.

3. Select Delete resource group.

   

4. Enter the resource group name. Then select Delete.

Next steps

Learn more about:

• From artifacts to models in MLflow
• Using Git with Azure Machine Learning
• Running Jupyter notebooks in your workspace
• Working with a compute instance terminal in your workspace
• Manage notebook and terminal sessions

This tutorial showed you the early steps of creating a model, prototyping on the same machine where the code resides. For your production training, learn how to use that training script on more powerful remote compute resources:

Train a model