Got "Connection error" by AzureOpenAI Chat Completions on Azure Kubernetes services, can anyone advise?

Question

Got "Connection error" by AzureOpenAI Chat Completions on Azure Kubernetes services, can anyone advise?

Ying, Jun 0

When I try to test the below Chat Completions codes, I got "Connection error".

https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/managed-identity

response = client.chat.completions.create( model="gpt-35-turbo-0125", # model = "deployment_name". messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Does Azure OpenAI support customer managed keys?"}, {"role": "assistant", "content": "Yes, customer managed keys are supported by Azure OpenAI."}, {"role": "user", "content": "Do other Azure AI services support this too?"} ] )

Pavankumar Purilla 11,575 Reputation points Microsoft External Staff Moderator

2025-04-16T07:51:02.51+00:00

Hi Ying, Jun,

To troubleshoot the Connection error you're encountering with Azure OpenAI Chat Completions on Azure Kubernetes Services (AKS), I recommend testing the code first from a machine with public internet access, such as your local machine or an Azure VM outside of AKS. This will help determine if the issue is specific to your AKS environment or a more general connectivity problem with Azure OpenAI. If the code works from a machine with internet access, the next step is to verify the connectivity from your AKS environment to the Azure OpenAI endpoint.

Ensure that there are no firewall rules, network policies, or other network restrictions blocking outbound traffic from your AKS cluster. Additionally, confirm that the managed identity associated with your AKS cluster has the necessary permissions to access Azure OpenAI.

I hope this information helps. Thank you!
Pavankumar Purilla 11,575 Reputation points Microsoft External Staff Moderator

2025-04-17T05:44:33.1366667+00:00

Hi Ying, Jun,
Did you get any chance to check the response. Thank you!
Pavankumar Purilla 11,575 Reputation points Microsoft External Staff Moderator

2025-04-18T02:37:16.6866667+00:00

Hi Ying, Jun,
Just following up to see if you had a chance to review the above response. Thank you!