How to get best model from sweep job in azure ml sdk ?

Question

Hello everyone,

I have created a sweep job in SDK. Now, I want to get the best model from the sweep job and register it to workspace within the SDK. However I couldn't find any code in the documents about getting the best model from the sweep job. After registering the best model, I will continue with the deployment process. Can you please advice ? Here is my code :

from azure.ai.ml import command, Input
job = command(
    code="./pipeline_src",  # local path where the code is stored
    command="python main.py --data ${{inputs.data}} --iterations ${{inputs.iterations}} --learning_rate ${{inputs.learning_rate}} --depth ${{inputs.depth}}",
    environment="my-env:6",  
    inputs={
        "data": "my_data",
        "iterations": 1000,
        "learning_rate": 0.03,
        "depth": 6
    },
    compute="cpu_cluster",
    display_name="catboost-hyperparameter-tuning",
)


command_job_for_sweep = job(
        iterations= Choice(values=[500, 1000, 1500]),
        learning_rate= Choice(values=[0.001, 0.01, 0.1, 0.5]),
        depth= Choice(values=[4, 6, 8, 10])
)

sweep_job = command_job_for_sweep.sweep(
    sampling_algorithm=RandomSamplingAlgorithm(rule='random'),
    primary_metric="mae",
    compute="cpu_cluster",
    goal="minimize",
    max_total_trials=20,
    max_concurrent_trials=4
)

sweep_job.experiment_name = "catboost_sweep"

sweep_job = ml_client.jobs.create_or_update(sweep_job)

Answer 1

Hello,

Getting the Best Model from the Sweep Job:

In Azure Machine Learning, you can perform hyperparameter tuning using a sweep job. The sweep job explores different hyperparameter combinations to find the best-performing model.

Assuming you’ve already created a sweep job, you can retrieve the best model from it. Here’s how you can do it using the Azure Machine Learning Python SDK v2:

Assuming you have the sweep_job object

best_run = sweep_job.get_best_run_by_primary_metric()

best_model = best_run.register_model(model_name='best_model', model_path='outputs/model.pkl')

In the above code:

best_run represents the run with the best-performing hyperparameters.

model_path should point to the location of the trained model within the best run (e.g., 'outputs/model.pkl').

The register_model method registers the best model in your workspace.

Registering the Best Model:

Once you have the best model, you can register it in your Azure Machine Learning workspace. Registration allows you to track and version your models.

Here’s how you can register the best model:

from azureml.core import Workspace, Model

Load your workspace

workspace = Workspace.from_config()

Register the best model

model = Model.register(model_path='outputs/model.pkl', model_name='best_model', workspace=workspace)

In the above code:

model_path should be the same as the one used during the best model retrieval.

model_name is the name you want to give to your registered model.

workspace is your Azure Machine Learning workspace.

Remember to adjust the paths and names according to your actual setup. Once you’ve registered the best model, you can proceed with the deployment process.

I hope this info is helpful to you.

Best Regard,
Ruth Johnson
Welcome to ADP

Answer 2

Did you manage to solve it @Sena Aslan ? I am having the same issue