How databrick save ml model into Azure Storage Container?


I am trying to use mlflow package in databricks to save the model into Azure Storage.

The Script:


project = 'test'

model_version = 'v1.0.1'

model = {model training step}
prefix_model_path = os.path.join(abfss_path, project, model_version)

model_path = prefix_model_path

print(model_path) # abfss://

mlflow.sklearn.save_model(model, model_path)

The message is successfully save the model.

When I check the container and file does not exist, but I am able to load model with the same path. That mean the model file saved in databricks somewhere.

I want to know where is the model file in databricks, and how to save the model directly from databricks notebook to Azure Storage.


Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,670 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,034 questions
{count} votes

Accepted answer
  1. HimanshuSinha-msft 19,386 Reputation points Microsoft Employee

    Hello @Benny Lau ,Shui Hong - Group Office ,

    Thanks for the ask and welcome to Microsoft Q&A .

    As I understand the ask here is to where the model is saved and how you can save to the blob .

    As per the document here : [

    You have three option and I assume that your model file is getting stored in the DBFS on the Azure databricks cluster .

    User's image

    Databricks can save a machine learning model to an Azure Storage Container using the dbutils.fs module. This module provides a set of functions for interacting with the Databricks file system (DBFS) and Azure Blob Storage. Here is an example of how to save a model to an Azure Storage Container:

    1. First, you will need to mount the Azure Storage Container to DBFS, this can be done using the dbutils.fs.mount function.
        "": "OAuth",
        "": "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider",
        "": "<your-client-id>",
        "": "<your-client-secret>",
        "": "<your-tenant-id>/oauth2/token"
    1. Once the container is mounted, you can use the dbutils.fs.cp function to copy the model from the local file system to the mount point.

    dbutils.fs.cp("path/to/local/model", "/mnt/<your-mount-point>/model")

    1. You can also use method to save the model in the mounted container path"/mnt/<your-mount-point>/model")

    Note: Be sure to replace the placeholders in the above code with the appropriate values for your use case.

    0 comments No comments

0 additional answers

Sort by: Most helpful