Which Azure service to host this ML model

Question

Which Azure service to host this ML model

Julio Fontoura 20

Hi.

I need to execute this model https://github.com/obss/sahi upon an HTTP request. I will need between 32GB and 128GB of RAM (depending on the request). Also, I will only receive this request once or twice a week (they are not predefined dates). Each process may take a few hours.

I have discarded Azure Functions since it has 10min execution limit (for Consumption Plan), and because it scales horizontally. I do not need multiple instances of the Functions, instead I need a powerful machine to run the model.

I am trying to understand if I can achieve my needs with Azure Machine Learning, Azure Batch or Azure VM, considering I want something that doesn't stay up 24/7 charging costs. I need something that can run when needed, and then free resources after that. Cold start is not a concern.

Any advice is appreciated. Thanks in advance.

Ramr-msft 17,826 Reputation points

2023-05-30T14:08:06.33+00:00

@Julio Fontoura Thanks for the question. it may not be necessary to use Azure Batch for processing a single large image.

Azure Batch is designed for running large-scale parallel and batch computing jobs, where you need to process a large number of tasks or jobs simultaneously. In your case, you only have one task to process, which is a single large image.

Instead of using Azure Batch, you can consider using a single virtual machine (VM) with high CPU and memory resources to process the image. You can use Azure Virtual Machines to create a VM with the required specifications and then run your image processing code on that VM.

Once the processing is complete, you can save the result to Azure Blob Storage as you mentioned.

In summary, for your use case of processing a single large image, it may be more cost-effective and efficient to use a single high-spec VM instead of Azure Batch.

1 answer

Your answer

Ramr-msft 17,826 Reputation points

2023-05-30T14:08:06.33+00:00

@Julio Fontoura Thanks for the question. it may not be necessary to use Azure Batch for processing a single large image.

Azure Batch is designed for running large-scale parallel and batch computing jobs, where you need to process a large number of tasks or jobs simultaneously. In your case, you only have one task to process, which is a single large image.

Instead of using Azure Batch, you can consider using a single virtual machine (VM) with high CPU and memory resources to process the image. You can use Azure Virtual Machines to create a VM with the required specifications and then run your image processing code on that VM.

Once the processing is complete, you can save the result to Azure Blob Storage as you mentioned.

In summary, for your use case of processing a single large image, it may be more cost-effective and efficient to use a single high-spec VM instead of Azure Batch.

Answer 1

Based on your requirements, Azure Batch is a suitable service for hosting your machine learning model. Azure Batch allows you to run large-scale parallel and high-performance computing (HPC) applications without the need to manage and provision the underlying infrastructure.

Here's how Azure Batch aligns with your needs:

Execution with high RAM requirements: Azure Batch supports the execution of tasks on VMs with different configurations, including VMs with high RAM capacity. You can choose a VM configuration with 32GB or 128GB of RAM based on your needs.

Ad hoc and infrequent requests: Azure Batch is designed for running parallel and batch processing workloads, which aligns well with your infrequent requests. You can submit jobs to Azure Batch when you receive a request, and it will handle the scheduling and execution of those jobs on the specified VMs.

Resource optimization: Azure Batch allows you to specify the number and size of VMs to be provisioned for the execution of your tasks. You can scale up and down based on the RAM requirements and number of requests. After the tasks are completed, you can deallocate the VMs to free up resources and minimize costs.

Cost efficiency: Azure Batch offers cost optimization through on-demand VMs and low-priority VMs (preemptible VMs), which can significantly reduce costs for non-time-critical workloads. By using low-priority VMs, you can take advantage of surplus capacity at a reduced cost.

Considering your requirements and the need for a powerful machine that can be provisioned on-demand, Azure Batch is a suitable choice. It provides the flexibility to scale up resources when needed, run the model for several hours, and then deallocate the resources to minimize costs.

Julio Fontoura 20 Reputation points

2023-05-30T14:54:45.87+00:00

@VasimTamboli Thanks for the detailed information.

I just want to give an additional information, to clarify what I mean by "process may take a few hours".

Basically, the request will tell me the URL of a big image (GBs size) in Azure Blob Storage, and that is the image to be processed in the model. The HTTP request will not need to wait the process to end. At the end I'll save the result of the model in Azure Blob Storage also.

For now we are not splitting the image, that's why it's a single big execution, and not a batch or multiple jobs.

Does it still make sense using Azure Batch? Considering I need 1 VM running 1 job?

Thanks in advance.

Share via

Which Azure service to host this ML model

1 answer

Your answer