After deploying an Azure ML model to a container instance, call to the model fails when using the code provided in the "Consume" section of the endpoint (Python and C#).

Question

I have trained a model in Azure Auto-ML and deployed the model to a container instance.

Now when I am trying to use the Python code provided in the Endpoint's "Consume" section I get the following error:

The request failed with status code: 502
Content-Length: 55
Content-Type: text/html; charset=utf-8
Date: Mon, 07 Mar 2022 12:32:07 GMT
Server: nginx/1.14.0 (Ubuntu)
X-Ms-Request-Id: 768c2eb5-10f3-4e8a-9412-3fcfc0f6d648
X-Ms-Run-Function-Failed: True
Connection: close

---

JSONDecodeError Traceback (most recent call last)
<ipython-input-1-6eeff158e915> in <module>
48 # Print the headers - they include the requert ID and the timestamp, which are useful for debugging the failure
49 print(error.info())
---> 50 print(json.loads(error.read().decode("utf8", 'ignore')))

/anaconda/envs/azureml_py36/lib/python3.6/json/init.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
352 parse_int is None and parse_float is None and
353 parse_constant is None and object_pairs_hook is None and not kw):
--> 354 return _default_decoder.decode(s)
355 if cls is None:
356 cls = JSONDecoder

/anaconda/envs/azureml_py36/lib/python3.6/json/decoder.py in decode(self, s, _w)
337
338 """
--> 339 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
340 end = _w(s, end).end()
341 if end != len(s):

/anaconda/envs/azureml_py36/lib/python3.6/json/decoder.py in raw_decode(self, s, idx)
355 obj, end = self.scan_once(s, idx)
356 except StopIteration as err:
--> 357 raise JSONDecodeError("Expecting value", s, err.value) from None
358 return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

If I use C# code provided in the Endpoint's "Consume" section I get the following error:

The request failed with status code: BadGateway
Connection: keep-alive
X-Ms-Request-Id: 5c3543cf-29ac-46a3-a9fb-dcb6a0041b08
X-Ms-Run-Function-Failed: True
Date: Mon, 07 Mar 2022 12:38:32 GMT
Server: nginx/1.14.0 (Ubuntu)

'<=' not supported between instances of 'str' and 'int'

Could you please help me with this issue? I am not sure what do to if Microsoft's provided code is erroring out, don't know what else to do.

The Python code I am using is:

import urllib.request
import json
import os
import ssl

def allowSelfSignedHttps(allowed):
    # bypass the server certificate verification on client side
    if allowed and not os.environ.get('PYTHONHTTPSVERIFY', '') and getattr(ssl, '_create_unverified_context', None):
        ssl._create_default_https_context = ssl._create_unverified_context

allowSelfSignedHttps(True) # this line is needed if you use self-signed certificate in your scoring service.

data = {
    "Inputs": {
        "data":
        [
            {
                "SaleDate": "2022-02-08T00:00:00.000Z",
                "OfferingGroupId": "0",
                "week_of_year": "7",
                "month_of_year": "2",
                "day_of_week": "1"
            },
        ]
    },
    "GlobalParameters": {
        "quantiles": "0.025,0.975"
    }
}

body = str.encode(json.dumps(data))

url = 'http://4a0427c2-30d4-477e-85f5-dfdfdfdfdsfdff623f.uksouth.azurecontainer.io/score'
api_key = '' # Replace this with the API key for the web service
headers = {'Content-Type':'application/json', 'Authorization':('Bearer '+ api_key)}

req = urllib.request.Request(url, body, headers)

try:
    response = urllib.request.urlopen(req)

    result = response.read()
    print(result)
except urllib.error.HTTPError as error:
    print("The request failed with status code: " + str(error.code))

    # Print the headers - they include the requert ID and the timestamp, which are useful for debugging the failure
    print(error.info())
    print(json.loads(error.read().decode("utf8", 'ignore')))

The C# code I am using is:

using System;
using System.Collections.Generic;
using System.IO;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Threading.Tasks;
using Newtonsoft.Json;

namespace MLModelAPICall
{
    class Program
    {
        static void Main(string[] args)
        {
            InvokeRequestResponseService().Wait();
        }

        static async Task InvokeRequestResponseService()
        {
            var handler = new HttpClientHandler()
            {
                ClientCertificateOptions = ClientCertificateOption.Manual,
                ServerCertificateCustomValidationCallback =
                        (httpRequestMessage, cert, cetChain, policyErrors) => { return true; }
            };
            using (var client = new HttpClient(handler))
            {
                // Request data goes here
                var scoreRequest = new
                {
                    Inputs = new Dictionary<string, List<Dictionary<string, string>>>()
                    {
                        {
                            "data",
                            new List<Dictionary<string, string>>()
                            {
                                new Dictionary<string, string>()
                                {
                                    {
                                        "SaleDate", "2022-02-08T00:00:00.000Z"
                                    },
                                    {
                                        "OfferingGroupId", "0"
                                    },
                                    {
                                        "week_of_year", "7"
                                    },
                                    {
                                        "month_of_year", "2"
                                    },
                                    {
                                        "day_of_week", "1"
                                    }
                                }
                            }
                        }
                    },
                    GlobalParameters = new Dictionary<string, string>()
                    {
                        {
                            "quantiles", "0.025,0.975"
                        }
                    }
                };


                const string apiKey = ""; // Replace this with the API key for the web service
                client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", apiKey);
                client.BaseAddress = new Uri("http://4a0427c2-30d4-477e-85f5-xxxxxxxxxxxxx.uksouth.azurecontainer.io/score");

                // WARNING: The 'await' statement below can result in a deadlock
                // if you are calling this code from the UI thread of an ASP.Net application.
                // One way to address this would be to call ConfigureAwait(false)
                // so that the execution does not attempt to resume on the original context.
                // For instance, replace code such as:
                //      result = await DoSomeTask()
                // with the following:
                //      result = await DoSomeTask().ConfigureAwait(false)

                var requestString = JsonConvert.SerializeObject(scoreRequest);
                var content = new StringContent(requestString);

                content.Headers.ContentType = new MediaTypeHeaderValue("application/json");

                HttpResponseMessage response = await client.PostAsync("", content);

                if (response.IsSuccessStatusCode)
                {
                    string result = await response.Content.ReadAsStringAsync();
                    Console.WriteLine("Result: {0}", result);
                }
                else
                {
                    Console.WriteLine(string.Format("The request failed with status code: {0}", response.StatusCode));

                    // Print the headers - they include the requert ID and the timestamp,
                    // which are useful for debugging the failure
                    Console.WriteLine(response.Headers.ToString());

                    string responseContent = await response.Content.ReadAsStringAsync();
                    Console.WriteLine(responseContent);
                    Console.ReadLine();
                }
            }
        }
    }
}

Answer 1

After much more digging I found out that the "Consume" scripts provided with the endpoint are wrong (Python and C#) .

When making a call to the endpoint the GlobalParameters expects an integer value, but the provided scripts have wrapped the values in double quotes hence making it a string:

 },
 "GlobalParameters": {
     "quantiles": "0.025,0.975"
 }

If you are using Python to consume the model, when making call to the endpoint your GlobalParameters should be define as this:

 },
 "GlobalParameters": {
     "quantiles": [0.025,0.975]
 }

wrapped in square brackets

[0.025,0.975]

and not in double quotes "

I have also opened a ticket with microsoft so hopefully they will fix the code provided in the "consume" section of every endpoint