Azure Machine Learning Studios Deploy Model - Obtain logs

Question

I am attempting to deploy a model to an Endpoint in MS Azure Machine Learning studio but I get some errror.

2 Questions

1. How can we get the logs for a deployed model (See my attempts below)
2. How do models get loaded into the docker images as I suspect for some reason it didn't get copied into the docker image.

Step 1) Upload the model: https://learn.microsoft.com/en-us/azure/machine-learning/how-to-deploy-existing-model

Which is successful. I can see and download that my model is correct.

Step 2) Deploy to Azure Container Instances: https://learn.microsoft.com/en-us/azure/machine-learning/how-to-deploy-azure-container-instance

I use this code snippet to deploy my code

from azureml.core.model import InferenceConfig  
from azureml.core.webservice import AciWebservice  
from azureml.core.webservice import Webservice  
from azureml.core.model import Model  
from azureml.core.environment import Environment  

script_file_name = 'inference/score.py'  

inference_config = InferenceConfig(entry_script=script_file_name, environment=tf_env)  

aciconfig = AciWebservice.deploy_configuration(cpu_cores = 1,   
                                               memory_gb = 1,   
                                               tags = {'iris': "rh1832", 'type': "sklearn"},   
                                               description = 'sample service for iris')  

aci_service_name = 'rh1832-iris-demo'  
print(aci_service_name)  
aci_service = Model.deploy(ws, aci_service_name, [model], inference_config, aciconfig)  
aci_service.wait_for_deployment(True)  
print(aci_service.state)  

And my score.py inference script looks like

# ---------------------------------------------------------  
# Copyright (c) Microsoft Corporation. All rights reserved.  
# ---------------------------------------------------------  
import json  
import logging  
import os  
import pickle  
import numpy as np  
import pandas as pd  
import joblib  

try:  
    logger = logging.getLogger('azureml.automl.core.scoring_script')  
except:  
    pass  


def init():  
    global model  
    model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'sklearn_model')  
    path = os.path.normpath(model_path)  
    path_split = path.split(os.sep)  
    logger.info("Loading model from path.")  
    model = joblib.load(model_path)  
    logger.info("Loading successful.")  


def run(data):  
    try:  
        target_names = ['database', 'network', 'resource']  
        result = model.predict([data])  
        return json.dumps(target_names[result[0]])  
    except Exception as e:  
        print("error: " + str(e))  
        result = str(e)  
        return json.dumps({"error": result})  

After submitting the webservice to be deployed i get an error that just says its in crashloopbackoff state with no details for which part of the script failed.

Step 3) I attempted to debug the logs by follow this guide: https://learn.microsoft.com/en-us/azure/machine-learning/how-to-troubleshoot-deployment?tabs=azcli#dockerlog

1) Running script locally which I was able to successfully run score.py
2) Running docker image locally I didn't see the model in the folder azureml-environment-setup in the docker container:

root@09e7230839e3:/azureml-environment-setup# ls
environment_context.json log4j.properties mutated_conda_dependencies.yml send_conda_dependencies.py spark_cache.py

and also i didn't see the entrypoint for running the docker container. Is there any guide for running the docker image locally?

3) Finally I attempted to get the logs using Azure CLI and it only return an Null list it appeared

Answer 1

You can try https://learn.microsoft.com/en-us/azure/machine-learning/how-to-debug-visual-studio-code#debug-and-troubleshoot-deployments