Share via

Optimize model with Intel OneAPI on Azure ML

Karnik Kanojia 0 Reputation points
2024-07-14T14:51:14.0633333+00:00

I have already successfully deployed my deep learning model as an online endpoint using the following docs. Now I'm seeking to optimize the inference timings. The VM size is following Standard_DS3_v2 which belongs to DSv2 series. My questions are the following:

  1. Is there any Azure ML curated environment that has the Intel OneAPI build and I can stack my docker context upon it to optimize things?
  2. Are there other ways to optimize the things too?
Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,960 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Azar 22,870 Reputation points MVP
    2024-07-14T18:58:11.1133333+00:00

    Hi there Karnik Kanojia

    Thanks for using QandA platform

    For optimizing, you can create a custom Docker image with Intel OneAPI and use it as your base. Start with a Dockerfile using the intel/oneapi-basekit image, build and push it to Azure Container Registry, and configure your Azure ML endpoint to use this custom image.

    also, try model quantization, pruning, using more powerful VM instances like the NC series, optimizing your inference pipeline,

    If this helps kindly accept the response thanks much,

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.