AI-Foundry Tracing Issue

Question

AI-Foundry Tracing Issue

Daun (Daniel) Lee 190

Hi, I am testing AI Foundry Tracing like LangSmith and since I am using LangChain I am trying to Integration Tracing as guided here.

https://ai.azure.com/nextgen/r/s8RkQsLiTkCIVp2k-URInA,00_AI_RG,,KT-OpenAI,KT-OpenAI-project/docs/observability/how-to/trace-agent-framework

Question1 : This is my result. Why my result is all separate and per one LLM call?

Question 2: How can I change the name : "chat gpt-4.1-mini-2025-04-14"

This is what I did.

Install

pip install langchain-azure-ai azure-monitor-opentelemetry-exporter rich azure-monitor-opentelemetry --no-deps

tracer declaration

from langchain_azure_ai.callbacks.tracers import AzureAIOpenTelemetryTracer

conn=os.environ.get([APPLICATION_INSIGHTS_CONNECTION_STRING from AI Foundry])

print(conn)

azure_tracer = AzureAIOpenTelemetryTracer(

connection_string=conn,

enable_content_recording=True,

name="Weather information agent"

)

tracers = [azure_tracer]

Add to model (tried 1.0 version method in the docs but didn't work so tried 0.3 version way and at least the log appeared in the AI Foundry portal.)

import os

from langchain_openai import AzureChatOpenAI

llm = AzureChatOpenAI(

model=os.environ['MODEL_NAME'],

azure_deployment=os.environ["MODEL_NAME"],

azure_endpoint=os.environ["END_POINT"],

openai_api_version="2025-03-01-preview",

openai_api_key=os.environ["AZURE_OPENAI_API_KEY"],

callbacks=[azure_tracer]

)

llm.invoke("Hello")

My Results conversation_my2

conversation_my

Documentation Screenshot conversation

Anshika Varshney 9,905 Reputation points Microsoft External Staff Moderator

2025-11-25T02:53:01.4833333+00:00

Hi Daun (Daniel) Lee,

Thanks for sharing the Question.

I’m currently checking this issue on my side and reviewing the tracing behavior in AI Foundry. I’ll share an update as soon as I have more findings or a workaround.

Thankyou!
Daun (Daniel) Lee 190 Reputation points

2025-11-25T15:16:23.5633333+00:00

Hi @Anshika Varshney

I tried to change the deployment name of the model but it is disabled as below.

How to change it and apply it in Tracing? Thanks!
Anshika Varshney 9,905 Reputation points Microsoft External Staff Moderator

2025-11-26T07:13:19.5766667+00:00
Hi Daun (Daniel) Lee,

Thanks for the follow-up!

The deployment name becomes locked/disabled after the model is created, so you won’t be able to edit or rename it directly from the UI. This is expected behavior in Azure AI Foundry.

To apply a different deployment name in Tracing, you can use one of the following approaches:

1. Create a New Deployment (Recommended)

If you need a different deployment name, the only supported option is to create a new deployment of the same model with the desired name. Once created, you can select this new deployment in Tracing.

2.Update Your Application Code / Tracing Config

Instead of changing the deployment name, update the place where you reference it:

In your SDK code, update the deployment_name parameter.

In Tracing, pick the correct deployment from the dropdown (it only shows existing deployments).

Deployment names are treated as unique identifiers. After creation, Azure disables renaming to prevent breaking existing integrations.

Please let me know if there are any remaining questions.

Answer accepted by question author

2 additional answers

Your answer

Anshika Varshney 9,905 Reputation points Microsoft External Staff Moderator

2025-11-25T02:53:01.4833333+00:00

Hi Daun (Daniel) Lee,

Thanks for sharing the Question.

I’m currently checking this issue on my side and reviewing the tracing behavior in AI Foundry. I’ll share an update as soon as I have more findings or a workaround.

Thankyou!
Daun (Daniel) Lee 190 Reputation points

2025-11-25T15:16:23.5633333+00:00

Hi @Anshika Varshney

I tried to change the deployment name of the model but it is disabled as below.

How to change it and apply it in Tracing? Thanks!
Anshika Varshney 9,905 Reputation points Microsoft External Staff Moderator

2025-11-26T07:13:19.5766667+00:00

Hi Daun (Daniel) Lee,

Thanks for the follow-up!

The deployment name becomes locked/disabled after the model is created, so you won’t be able to edit or rename it directly from the UI. This is expected behavior in Azure AI Foundry.

To apply a different deployment name in Tracing, you can use one of the following approaches:

1. Create a New Deployment (Recommended)

If you need a different deployment name, the only supported option is to create a new deployment of the same model with the desired name. Once created, you can select this new deployment in Tracing.

2.Update Your Application Code / Tracing Config

Instead of changing the deployment name, update the place where you reference it:

In your SDK code, update the deployment_name parameter.

In Tracing, pick the correct deployment from the dropdown (it only shows existing deployments).

Deployment names are treated as unique identifiers. After creation, Azure disables renaming to prevent breaking existing integrations.

Please let me know if there are any remaining questions.

Answer 1

Anshika Varshney 9,905 Microsoft External Staff Moderator

Hi Daun (Daniel) Lee,

Thanks for sharing the details!

Separate traces per LLM call – This happens because llm.invoke() is treated as an individual run in LangChain, so AI Foundry logs each call separately. To group traces, wrap your logic in a parent span or use a full chain/agent instead of calling the raw model directly.

Changing the name – The name shown (e.g., chat gpt-4.1-mini-2025-04-14) comes from your Azure OpenAI deployment name. To update it, rename or recreate the deployment with the desired name in Azure.

For more guidance, check the official docs: Trace AI Agents in Azure AI Foundry

Please let me know if there are any remaining questions or additional details, I can help with, I’ll be glad to provide further clarification or guidance.

Anshika Varshney 9,905 Reputation points Microsoft External Staff Moderator

2025-11-26T08:02:14.38+00:00

Hi Daun (Daniel) Lee,

I’ve converted comment into an answer, could you please take a moment to mark it as Accepted? This helps others in the community with the same question find the solution more easily. Thankyou!

Answer 2

Adam Zachary 2,265

I’ve run into this before, and both issues come from how you’re calling the LLM.

You see separate traces because you’re calling llm.invoke() directly. LangChain treats each call as its own run, so Foundry logs each LLM call as a separate trace. If you want a single grouped trace, you need to wrap your logic in a parent span or call an actual chain/agent instead of the raw model.
The name “chat gpt-4.1-mini-2025-04-14” comes from the Azure OpenAI deployment name. Foundry is just showing whatever deployment name your code is using. To change it, rename or recreate the deployment with a different name.

Daun (Daniel) Lee 190 Reputation points

2025-11-25T03:42:30.99+00:00

Hi @Adam Zachary Thank you for sharing your experiences. In LangSmith, once a project name is injected as env variable, all the calls are grouped into one; Does AI Foundry use different method from this?
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Adam Zachary 2,265 Reputation points

2025-11-25T03:52:50.9833333+00:00

Unsure why I can't post my comment
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Adam Zachary 2,265 Reputation points

2025-11-25T03:55:02.29+00:00
LangSmith groups runs automatically when you set the project environment variable because its runtime manages the hierarchy for you. Azure AI Foundry works differently. It records each LLM invocation as an individual trace unless your code introduces a higher-level span or you are executing a full LangChain chain/agent that naturally produces a parent run.

In practice this means:

• A direct llm.invoke() call is captured as a standalone trace.

• Environment variables such as the LangSmith project value do not influence grouping inside AI Foundry.

• Grouping appears only when there is a parent span or a chain/agent structure that Foundry can anchor the child spans under.

To group multiple LLM calls into a single trace, you can either run them through a chain/agent or create a parent span yourself:

with azure_tracer.start_as_current_span("Weather-Agent-Run"):

llm.invoke("Hello") llm.invoke("Another call")

With this approach, AI Foundry treats the parent span as the root and organizes the individual LLM calls underneath it in one grouped trace.
Daun (Daniel) Lee 190 Reputation points

2025-11-25T03:55:30.99+00:00

@Adam Zachary For the concerning comment, would you send ******@naver.com instead for the time being? Thanks!
Adam Zachary 2,265 Reputation points

2025-11-25T03:57:08.21+00:00

They hash email accounts. You can find me on LinkedIn
Daun (Daniel) Lee 190 Reputation points

2025-11-25T04:01:44.6566667+00:00

Hi @Adam Zachary From Mobiz IT?
Adam Zachary 2,265 Reputation points

2025-11-25T04:03:56.9366667+00:00

Yes, that's me
Daun (Daniel) Lee 190 Reputation points

2025-11-25T04:25:25.2033333+00:00

@Adam Zachary Another concern is that unlike LangChain which gives individual LangSmith environment on the web freely, AI Foundry holds tracing within a Foundry Project, which is limited to be created multiple or unlimited times due to subscription quota of Azure.

So in Azure, the tracing should be per app of an organization not individually within a tenant. Am I understanding correctly?

Answer 3

Deleted

This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

Comments have been turned off. Learn more

Answer 4

Deleted

This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

Comments have been turned off. Learn more

Share via

AI-Foundry Tracing Issue

2 additional answers

Your answer