Databricks mlflow model serving networking

Question

Databricks mlflow model serving networking

Chammie Ho 51

Hi there, I'm trying to register a ML model to the mlflow model registry to be served from an Azure web app. I'm wondering if the databricks networking configuration will apply to the model api endpoint, as in, calls to the api from outside the VNET which the databricks is deployed to with private endpoints and disabled public access will be rejected and the call from within the vnet integrated web app will be successful?

Thank you!

romungi-MSFT 30,211 Reputation points Microsoft Employee

2022-09-07T08:39:06.027+00:00
@Chammie Ho I believe you have your databricks setup with a private endpoint and would like to call your mlflow endpoint from a webapp from within the same private endpoint?
Based on what I read from the documentation you would need to ensure your ML workspace also uses the same private endpoint and the deployed mlflow endpoint should then be able to accessible from your webapp.

To setup your Azure ML workspace with private endpoint you will have to satisfy all the criteria mentioned here

If the above is successful you should be able to submit experiments in your workspace and deploy a mlfow endpoint which is basically the steps mentioned here

While creating the endpoint using az ml online-endpoint you will have to ensure the YAML configuration of online endpoint sets the field public_network_access to disabled

While creating the deployment or setting the model that serves the endpoint the YAML configuration of managed online deployment sets the field egress_public_network_access to disabled
romungi-MSFT 30,211 Reputation points Microsoft Employee

2022-09-07T08:40:15.39+00:00

I would recommend using the studio UI for your initial deployment and if it works as expected you can try to use the CLI or SDK v2 for corresponding deployments. Thanks!!
Chammie Ho 51 Reputation points

2022-09-07T12:37:45.88+00:00

Thank you for your answer, I think I might have been ambiguous with my question, my ML model is actually created from within databricks and I'm planning to register it to the mlflow registry from within databricks (https://learn.microsoft.com/en-us/azure/databricks/applications/mlflow/model-serving) then enabling model serving, after which I should be able to call the model with the url <databricks-instance>/model/<registered-model-name>/<model-version>/invocations, my question is whether this url will have the same restrictions as the databricks where it resides. In the documentation I can see that the authentication of the API is still using databricks PAT, but I can't find information for IP restrictions.
PRADEEPCHEEKATLA-MSFT 56,471 Reputation points Microsoft Employee

2022-09-15T05:58:50.017+00:00
Hello @Chammie Ho ,

Following up to see if the below suggestion was helpful. And, if you have any further query do let us know.

------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you.

romungi-MSFT 30,211 Reputation points Microsoft Employee

2022-09-07T08:39:06.027+00:00

@Chammie Ho I believe you have your databricks setup with a private endpoint and would like to call your mlflow endpoint from a webapp from within the same private endpoint?
Based on what I read from the documentation you would need to ensure your ML workspace also uses the same private endpoint and the deployed mlflow endpoint should then be able to accessible from your webapp.

To setup your Azure ML workspace with private endpoint you will have to satisfy all the criteria mentioned here

If the above is successful you should be able to submit experiments in your workspace and deploy a mlfow endpoint which is basically the steps mentioned here

While creating the endpoint using az ml online-endpoint you will have to ensure the YAML configuration of online endpoint sets the field public_network_access to disabled

While creating the deployment or setting the model that serves the endpoint the YAML configuration of managed online deployment sets the field egress_public_network_access to disabled
romungi-MSFT 30,211 Reputation points Microsoft Employee

2022-09-07T08:40:15.39+00:00

I would recommend using the studio UI for your initial deployment and if it works as expected you can try to use the CLI or SDK v2 for corresponding deployments. Thanks!!
Chammie Ho 51 Reputation points

2022-09-07T12:37:45.88+00:00

Thank you for your answer, I think I might have been ambiguous with my question, my ML model is actually created from within databricks and I'm planning to register it to the mlflow registry from within databricks (https://learn.microsoft.com/en-us/azure/databricks/applications/mlflow/model-serving) then enabling model serving, after which I should be able to call the model with the url <databricks-instance>/model/<registered-model-name>/<model-version>/invocations, my question is whether this url will have the same restrictions as the databricks where it resides. In the documentation I can see that the authentication of the API is still using databricks PAT, but I can't find information for IP restrictions.
PRADEEPCHEEKATLA-MSFT 56,471 Reputation points Microsoft Employee

2022-09-15T05:58:50.017+00:00

Hello @Chammie Ho ,

Following up to see if the below suggestion was helpful. And, if you have any further query do let us know.

------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you.

Answer 1

Hello @Chammie Ho ,

Thanks for the question and using MS Q&A platform.

I'm wondering if the databricks networking configuration will apply to the model api endpoint

Yes, it will. The single node cluster on which the model is hosted (classic), is deployed in data plane and will have a private IP.

I should be able to call the model with the url <databricks-instance>/model/<registered-model-name>/<model-version>/invocations, my question is whether this url will have the same restrictions as the databricks where it resides

I believe, this should work as long as you have not defined any IP access list. The PAT will let you authenticate.

but I can't find information for IP restrictions

IP access lists - Azure Databricks | Microsoft Learn

Hope this will help. Please let us know if any further queries.

------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
Want a reminder to come back and check responses? Here is how to subscribe to a notification
If you are interested in joining the VM program and help shape the future of Q&A: Here is jhow you can be part of Q&A Volunteer Moderators

Chammie Ho 51 Reputation points

2022-09-16T11:57:57.697+00:00

Thank you for your confirmation!

Databricks mlflow model serving networking

0 additional answers

Activity