Hi, I'm trying to implement a Python app with llms served from Azure AI Foundry. I have the following code:
from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential
project_connection_string = "<project-connection-str>"
project = AIProjectClient.from_connection_string(
conn_str=project_connection_string, credential=DefaultAzureCredential()
)
chat = project.inference.get_chat_completions_client(connection_name="<connection-name>")
response = chat.complete(
model="<model-name>",
messages=[
{"role": "system","content": "You are an AI assistant that speaks like a techno punk rocker from 2350. Be cool but not too cool. Ya dig?",},
{"role": "user", "content": "Hey, can you help me with my taxes? I'm a freelancer."},
],
)
print(response.choices[0].message.content)
The error occurs during the chat.complete() method, and I don't know how to fix it. I want to connect through Foundry like this so I can use both open-source or openai models, instead of using AzureOpenAI objects (which works)
The following alternative results in the same error:
from azure.ai.inference import ChatCompletionsClient
from azure.core.credentials import AzureKeyCredential
# Datos de conexión
endpoint = "<endpoint>" # Cambia por el tuyo
api_key = "<api-key>" # Reemplaza con tu clave real
# Crear cliente de inferencia directamente
chat = ChatCompletionsClient(endpoint=endpoint, credential=AzureKeyCredential(api_key))
# Hacer la consulta al modelo específico
response = chat.complete(
model="<model-name>", # Asegúrate de usar el nombre exacto del deployment
messages=[{"role": "user", "content": "¿Cómo mejorar mi productividad?"}]
)
# Imprimir la respuesta
print(response.choices[0].message.content)
How can I fix this?