Is onnx the best way to deal with azure iot edge Machine Learning models?

Question

I am working on Azure IOT Edge. I have three models.
a)yolov3 object detection model in .weights format
b)Resnet classfication model
c)VGG16 classification model in .h5 format.

I converted them to onnx and using onnx runtime inferenced them and wrote the necessary scoring scripts.

I wanted to know how do I use these models in original format without converting them to onnx or is onnx the best way for iot edge modules?

Answer 1

Hi @Anurag Shelar it is not mandatory that you convert your models to onnx. Check this doc:

• Open-source integration with Azure Machine Learning projects

"You can train, deploy, and manage the end-to-end machine learning process in Azure Machine Learning by using open-source Python machine learning libraries and platforms. Use development tools, like Jupyter Notebooks and Visual Studio Code, to leverage your existing models and scripts in Azure Machine Learning."

Thanks.

Answer 2

If you didn't covert the models to ONNX then the respective inference engines will need to be included in your IOT-Edge module to run the inference.

ONNX provides the common format to represent the NN models from different training frameworks and execute using the same runtime, i.e. onnxruntime, on various edge device platforms. The same base docker image can be used to run the ONNX models across x86, arm64 devices with different accelerators like GPUs and VPUs.