How to generate an ONNX model from an Azure AI Machine Learning Studio Instance Segmentation model

Question

How to generate an ONNX model from an Azure AI Machine Learning Studio Instance Segmentation model

Richard Smith 20

I have created a Polygon (Instance Segmentation) Data Set in Machine Learning Studio.

I have labelled and approved numerous objects in numerous images.

I have generated a Model from the Data Set - which may be downloaded as a PyTorch file (plus numerous other files).

I need to generate an ONNX file but cannot find any way of doing that in Machine Learning Studio.

I have attempted to convert the .pt file downloaded from Azure using VS Code but I cannot find a way to get PyTorch to accept the .pt file.

All of the explanations I have found in the Azure "help" documentation load a pre-trained model from torch.models. All of the "help" documentation related to conversion to ONNX within Machine Learning Studio either references components that do not exist or the links themselves are broken.

I require the ONNX file for use in a C# Windows Desktop application.

1 answer

Your answer

Answer 1

SriLakshmi C 6,010 Microsoft External Staff Moderator

Hello Richard Smith,

To generate an ONNX model from your Azure AI Machine Learning Studio instance segmentation model, you will first need to convert your PyTorch model file (.pt) to the ONNX format. Here’s a general approach you can follow:

Ensure you have the necessary libraries installed by running

pip install torch onnx

Retrieve your .pt model file and any additional definition files from Azure ML Studio.

You need to load your PyTorch model in your Python environment. Use the torch.load() function to load the model from the .pt file.

If your model fails to load directly, try an alternative approach by exporting it within ML Studio using the "Convert to ONNX" step in your pipeline before downloading.

Before exporting, ensure that your model is set to evaluation mode by calling model.eval() or model.train(False). This step is crucial as layers like dropout and batch normalization function differently during training and inference.

Utilize the torch.onnx.export() function to convert your model to the ONNX format. Specify the model, a dummy input tensor matching the model's input shape, and the desired filename for the output ONNX file.

Once the export function is executed, the model.onnx file should be available in your working directory.

If you experience issues loading the .pt file, verify that it is compatible with your PyTorch version and that the correct model architecture is defined in your code.

After successfully generating the ONNX file, you can integrate it into your C# Windows Desktop application.

please refer this Convert your PyTorch model to ONNX format, Train a model with PyTorch and export to ONNX.

I hope this helps. And, if you have any further query do let us know.

Thank you!

Richard Smith 20 Reputation points

2025-03-17T02:07:23.7366667+00:00

Hi SC, thanks for the reply.

Unfortunately there is no "Convert to ONNX" step in the pipeline designer. Otherwise I would have used it.

I have attemted to load the .pt file that was generated, both within Azure and after downloading it to my hard drive. In both cases I hit the same issue:

Exception has occurred: UnpicklingError - Weights only load failed.

The option is to run torch.load(...) with weight_only= true, if I do that then it fails on the following line:

model.eval() with the error - Exception has occurred: AttributeError - 'dict' object has no attribute 'eval'

Since I do not seem to be able to load the .pt file I cannot get to the export stage and since the pipeline does not include a "Convert to ONNX" step, I cannot use that either.

Regards

Richard
SriLakshmi C 6,010 Reputation points Microsoft External Staff Moderator

2025-03-18T10:04:56.83+00:00
Hi Richard Smith,

The .pt file downloaded from Azure ML Studio likely contains only the model weights rather than the complete PyTorch model. As a result, torch.load() returns a dictionary instead of a model object, causing the error: AttributeError: 'dict' object has no attribute 'eval'.

Rebuild the Model Before ONNX Export

1.Determine the Model Architecture

Determine the Model Architecture Since Azure ML Studio does not include the model architecture with the .pt file, you must manually recreate the model class.

If you utilized a prebuilt model from Azure ML Studio, refer to the documentation for its corresponding PyTorch model architecture, Train PyTorch models at scale with Azure Machine Learning SDK.

2.Define and Load the Model

Once you have identified the model architecture, define the same model structure in Python:

import torch import torch.onnx # Define the model architecture (this must match the one used during training) class MyInstanceSegmentationModel(torch.nn.Module): def __init__(self): super(MyInstanceSegmentationModel, self).__init__() # Define the layers as per the training script # Example: self.conv1 = torch.nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1) def forward(self, x): # Define forward pass return x # Modify this based on actual architecture # Initialize the model model = MyInstanceSegmentationModel() # Load the weights checkpoint = torch.load("model.pt", map_location=torch.device("cpu")) if isinstance(checkpoint, dict) and "model_state_dict" in checkpoint: model.load_state_dict(checkpoint["model_state_dict"]) else: model.load_state_dict(checkpoint) # Directly load if it's just state_dict # Set the model to evaluation mode model.eval()

3.Convert to ONNX

Once the model is successfully reconstructed and weights are loaded, you can convert it to ONNX:

# Create a dummy input matching the model's expected input shape dummy_input = torch.randn(1, 3, 256, 256) # Modify this based on your input shape # Export to ONNX torch.onnx.export(model, dummy_input, "model.onnx", opset_version=11, input_names=["input"], output_names=["output"]) print("ONNX model exported successfully!")

If the above approach does not work, try the following steps:

Use map_location=torch.device("cpu") in torch.load() to ensure compatibility with your device.

Print checkpoint.keys() to check if key names differ (e.g., state_dict instead of model_state_dict).

If load_state_dict() fails due to key mismatches, adjust the keys using:
model.load_state_dict({k.replace("module.", ""): v for k, v in checkpoint.items()}, strict=False)

If you are uncertain about the input shape, print the model structure and examine the forward() method.

Thank you!
SriLakshmi C 6,010 Reputation points Microsoft External Staff Moderator

2025-03-19T07:34:36.7066667+00:00

Hi Richard Smith,

Following up to see if the above suggestion was helpful. And, if you have any further query do let us know.

Thank you!
Richard Smith 20 Reputation points

2025-03-20T00:06:49.23+00:00

Hi SC, thanks for the reply.

Unfortunately it does not help.

You say that Azure does not include the model architecture and that I must determine this myself. I assume this is to be represented by the "MyInstanceSegmentationModel" structure?

Where to I find the necessary information to detemine the model architecture? What is the structure of MyInstanceSegmentationModel? Why does Azure not include the model architecture since it has all of the necessary details?

Please keep in mind that what I have done so far is to use Machine Learning Studio web interface to create a DataSet, upload an assortment of images and label them. I generated the model using the "Image Instance Segmentation Pipeline" component in the pipeline designer.

I am completely unfamiliar with python and am attempting to learn as I go.

I do not understand why Azure cannot provide a simple step-by-step tutorial for Image Instance Segmentation as they do for other trivial AI models.

I also do not understand why Azure Custom Vision can provide a straight forward user interface which includes all of the processes necessary to create a dataset, and train and export a model whilst Azure Machine Leanring Studio presents a convoluted, disconnected interface that does not include the tools necesssary to complete what should be a simple task.

Regards

Richard
JAYA SHANKAR G S 4,035 Reputation points Microsoft External Staff Moderator

2025-03-21T12:03:08.4066667+00:00
Hi @Richard Smith ,

Below are some pointers you may need to check.

Here, it's documented algorithms are supported in ONNX, please check this and confirm your algorithm.

If you are doing distributed training, then ONNX support are not currently supported in the distributed training mode.

Next, when you are submitting the job, you have an option ONNX normalization under additional configuration settings

and next

Try this option and check for ONXX model output in the best run.

Please add the screenshot of the files available in model output after your training job completed so that I can give you a different code where you no need to add explicitly give the architecture.

Thank you
JAYA SHANKAR G S 4,035 Reputation points Microsoft External Staff Moderator

2025-03-26T04:51:49.56+00:00

Hello @Richard Smith ,

I reached out to the internal product team, will let you know once I get more details on it.

Thank you

Share via

How to generate an ONNX model from an Azure AI Machine Learning Studio Instance Segmentation model

1 answer

Your answer