Monitor Kubernetes Online Endpoint inference server logs

APPLIES TO: Azure CLI ml extension v2 (current) Python SDK azure-ai-ml v2 (current)

To diagnose online issues and monitor Azure Machine Learning model inference server metrics, we usually need to collect model inference server logs.

AKS cluster

In AKS cluster, you can use the built-in ability to collect container logs. Follow the steps to collect inference server logs in AKS:

1. Go to the AKS portal and select Logs tab

   

2. Click Configure Monitoring to enable Azure Monitor for your AKS. In the Advanced Settings section, you can specify an existing Log Analytics or create a new one for collecting logs.

   

3. After about 1 hour for it to take effect, you can query inference server logs from AKS or Log Analytics portal.

   

4. Query example:

       let starttime = ago(1d);
       ContainerLogV2
       | where TimeGenerated > starttime
       | where PodName has "blue-sklearn-mnist"
       | where ContainerName has "inference-server"
       | project TimeGenerated, PodNamespace, PodName, ContainerName, LogMessage
       | limit 100
   

Azure Arc enabled cluster

In Arc Kubernetes cluster, you can reference the Azure Monitor document to upload logs to Log Analytics from your cluster by utilizing Azure Monitor Agent