Loading multiple models in an online fully managed endpoint

Question

Hey there!

I am creating an endpoint which chains inference from different models. At the moment I solve the loading of them by doing:

# define models paths
artifacts_dir = Path(os.getenv("AZUREML_MODEL_DIR")) / "artifacts"
model_file_path = "runwayml/stable-diffusion-v1-5"
gfpgan_model_file_path = artifacts_dir / "gfpgan" / "gfpgan_v1.4.pth"
controlnet_model_file_path = artifacts_dir / "controlnet" / "scribble_sd15"
embedding_file_path = (
    artifacts_dir / "stable_diffusion" / "sd15_journal_sketch_inversion.bin"
)

inference = Inference(
        model_file_path=model_file_path,
        gfpgan_model_file_path=gfpgan_model_file_path,
        enable_xformers=False,
        controlnet_type="controlnet",
        controlnet_model_file_path=controlnet_model_file_path,
        embedding_file_path=embedding_file_path,
)

The problem with this, is that I have all the models in a single artifact folder registered as a Model in the ML workspace. This works, but if I want to change or update only one of the models, I would need to re-upload the whole folder which is quite expensive in terms of storage and time.

I found an example using the python SDK v2:

from azureml.core.model import Model

def init():
    global model_1, model_2
    # Here "my_first_model" is the name of the model registered under the workspace.
    # This call will return the path to the .pkl file on the local disk.
    model_1_path = Model.get_model_path(model_name='my_first_model')
    model_2_path = Model.get_model_path(model_name='my_second_model')
    
    # Deserialize the model files back into scikit-learn models.
    model_1 = joblib.load(model_1_path)
    model_2 = joblib.load(model_2_path)

And I was wondering if there's anything similar for SDK v2. I tried to find information on the net, but couldn't find any.

If you have any lead would be greatly appreciated! Thank you so much!

Answer 1

@Pietro Bolcato Thanks, The Model.get_model_path method is used to retrieve the local file path of a registered model in both SDK v1 and v2. You can use this method to load individual models from your workspace without having to re-upload the entire folder. Here are the latest Azure ML Samples using the SDK v2.