Azure ML - How to do batch scoring if the workspace is using CMK (Customer Managed Keys)

Question

Hi,

As per the MS documentation if the workspace use CMK (Customer Managed Keys) , we can't create a Batch Endpoint that can be called to run Batch prediction. In this case what are the alternatives ? We have a use case where we need to do run a forecasting model every day on batch data. Thank you !

Answer 1

Hello @Chettiyattil Dharman, Mithun

Thanks for reaching out to us, if you are using Azure Machine Learning and your workspace is configured with Customer Managed Keys (CMK), you can still perform batch scoring using the following options:

Use Azure Data Factory: You can use Azure Data Factory to trigger a Python script that runs the batch scoring using Azure ML. The Python script can load the model and perform batch scoring on the input data. This approach requires some coding and integration with Azure Data Factory.

Use Azure Functions: You can also use Azure Functions to trigger the batch scoring process. Azure Functions can be triggered by a timer or a webhook, and can load the model and perform the batch scoring on the input data. This approach requires some coding and integration with Azure Functions.

Use Azure Logic Apps: Azure Logic Apps is a visual workflow automation tool that can be used to orchestrate the batch scoring process. You can create a Logic App that triggers the batch scoring process, waits for the process to complete, and then sends an email with the results. This approach requires some configuration and integration with Azure Logic Apps.

Use Azure Synapse Analytics: If you are already using Azure Synapse Analytics for your data processing needs, you can use it to trigger the batch scoring process. You can create a Synapse pipeline that runs a Python script or a notebook to perform the batch scoring on the input data. This approach requires some configuration and integration with Azure Synapse Analytics.

Note that each of these options requires some level of coding or configuration, so you should choose the one that best fits your skills and requirements. Additionally, you should consider the cost implications of each option, as some may be more expensive than others depending on your usage patterns.

I hope this helps, do let me know if you have any questions. Thanks.

Regards,

Yutong