Prompt Flow: need Trace in addition to Output

Question

The output of a node in Prompt Flow can be accessed using ${classify_with_llm.output} which yields the "output" key from Output:

However, I need to programmatically access the Trace also (also OK if it must be done through some logging tool external to Prompt Flow):

Thanks!

Answer 1

Hi Saurabh, I realised that an easier workaround is to just download all the flow artefacts from blob storage after completion of the run. Thanks nonetheless!

Answer 2

Hi @Daniel von Eschwege
Welcome to Microsoft Q&A! Thanks for posting the question.

I am checking this on internally with the products team. Meanwhile, can you please check if you can use the Promptflow library to get the trace information.

I will let you know once I get any more information on the same. Thanks Saurabh

Answer 3

Daniel von Eschwege Thank you for sharing your solution. I have posted an answer on your behalf, which will benefit other community members as well. Please accept the answer when you can.

Hi Saurabh, I realised that an easier workaround is to just download all the flow artefacts from blob storage after completion of the run. Thanks nonetheless!

Thanks

Saurabh

---

Please 'Accept as answer' and Upvote if it helped so that it can help others in the community looking for help on similar topics.

Answer 4

Hi Saurabh, After some more consideration, I realised that this doesn't really solve the problem. For example, in use cases where the token count is required for live billing, I need to at the very least get the token count returned when calling a LLM, not to mention everything else that is normally returned by a LLM call. Rather strange imo that all this info is readily available under Trace, but cannot be accessed as input for another node.

Answer 5

Hi @Daniel von Eschwege ,

I consulted with the products team and found out that there is no way to access this data currently. The only option for users is to examine the system output or trace after a run. The products team is aware of this issue and has added it to their internal backlog as this feature has been requested by several customers. They are also working on improving the trace capability in prompt flow, as well as providing a programmatic way to get the output/metrics/trace of the flow and node level. I apologize for not being able to give you an ETA at this time, but please stay tuned for public updates on this matter.

Thanks

Saurabh