Azure AI Foundry > Ability to render Images Directly in Chat?

Question

Hi!

I am creating a prompt flow based on a chat experience that I developed in python.

It's a simple RAG experience that pulls an image based on the user prompt. For example, if the user were to say "Show me all of our hero components" - the tool will analyze a JSON file with all image descriptions and present the images to the user directly in the chat window.

I am having an issue displaying images directly in the chat flow window and I am not sure what I am doing wrong. It looks like it will only present the base-64 and not render an actual image or images.

I am very new to azure, so maybe there is something simple I am missing. My end goal is to be able to launch this through co-pilot or teams so my company can use.

Any guidance would be awesome. Thanks so much in advance!

Answer 1

Hi tonytonytony

Azure OpenAI does not show images in chat yet like OpenAI, but you can interact with image and summarize them using Vision enable models like GPT 4.

You can already pull mage URLs when you add your own data feature instead of rendering the images.

You might succeed by customizing custom chat app but not sure on it yet as image rendering is not natively supported

You might need to modify the backend code on custom chat to parse the image URLs from JSON part and render smaller resolution of it to show it on webapp, else prompt users the open the image URL instead.

Attached Add your own data for python for experimentation

Hope the pointers helps.

Thank you.