This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
I have deployed a model to an endpoint but the predictions that come back do not match what is output from the scored model in the experiment.
The model is a Multiclass Neural Network for classifying emails - it takes approx. 1,000 elements of text and puts them into one of about 18 possible categories.
Reviewing the scored model outputs in the experiment, the accuracy is reasonable; however once deployed it always predicts any input into a single category with near 100% confidence. The input is correctly formatted (this is validated in the model output) so I'm assuming something is misconfigured in the model deployment, but I'm not seeing any indication as to what.
Appreciate any help from MSFT in investigating this issue.
Cheers, James
anonymous user Thanks for the question. There are multiple ways to create a multiclass DNN on various AzureML products. Is this from some tensorflow or pytorch code, or from the AML Designer? Screenshots or sample notebooks might help a lot.
Hi @Ramr-msft -
The model was created via the AML Designer. The deployed model outputs were collected from the "Tune Model Hyperparameters" outputs ("trained_model_outputs") folder and published to an online endpoint. The score.py has been modified to handle the inputs being sent via endpoint rather than batch in the experiment but I can verify from the scoring output that the model is getting the data in the correct format & structure.. I'm verifying the results against the outputs from the "Score Model" module.
It looks like something is going wrong in the ScoreModelModule() in score.py as it's not interpreting the input correctly, but I'm not sure how I can diagnose what the isse might be.
Thanks, James
anonymous user Thanks for the details. We have forwarded to the product team to check on this and will update on the same.
Hi @Ramr-msft - have you heard anything?
Cheers, James
anonymous user was there any kind of normalization / transforms applied during training that were maybe not applied during inferencing?.
Please share details of your experiment and issue from the ml.azure.com portal for a service engineer to lookup the issue from the back-end? This option is available from the top right hand corner of the portal by clicking the smiley face, Please select the option Microsoft can email you about the feedback along with a screen shot so our service team can lookup and advise through email.
Thanks @Ramr-msft - I have done so (after some re-evaluation on my side to make sure there were no obvious issues)
There are a whole bunch of transformations but we accounted for that in the score.py to make sure the input matched & this is verifiable in the model output.
Is there any way of tracking the engagement? As it's not a formal support request I don't get any tracking emails or contact.
Cheers, James
Hi @Ramr-msft - any update on how we can get Microsoft support with this? It's blocking a customer deployment of the model. Can you raise a one time support request or similar so I can open a ticket?
We are a Gold Partner so hopefully that can expedite getting help.
Thanks, James
Hi James, we're checking if your issue here is resolved. If it's not, can you please go to ML Studio and click the '?' icon on your top right and then 'New support request'? You'll be routed to the Azure Portal
New Support Request screen, and based on your description of the problem above, you can choose "Designer" as the Problem type and "Problem of deploying real-time endpoints in Designer" as Problem subtype.
Then you will receive an email from our support engineer team and you can discuss details to resolve the issue.
Sign in to comment