Open AI model to generate image as output based on input image

Question

I have recently started using azure open AI service and i have a scenario where i have to send a image to open ai and get the image as output with proper tags. For example we have a image of a park and we ask open AI to mark benches in the park and return that image as a response. Currently gpt4o model is analyzing image and providing text output whereas i need as a image as output.

Answer 1

Hi there Jyothsna

Thanks for using QandA platform.

I guess the Azure OpenAI Service currently focuses on text generation and analysis, and while GPT-4 can analyze images and provide text-based responses, it does not natively support generating or modifying images directly as output. but, you can do this by combining Azure services with a computer vision model capable of image processing.

https://learn.microsoft.com/en-us/azure/ai-services/computer-vision/

https://learn.microsoft.com/en-us/azure/ai-services/computer-vision/quickstarts-sdk/client-library?pivots=programming-language-python&tabs=windows%2Cvisual-studio

https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-python?tabs=managed-identity%2Croles-azure-portal%2Csign-in-azure-cli&pivots=blob-storage-quickstart-scratch

If this helps kindly accept the answer thanks much.

Answer 2

Open AI:

i see that chat GPT can generate image outputs for input image. Azure open ai will also have this feature in the future?

Azure Vision Service:

Azure's Computer Vision service detect the objects in images but it just returns coordinates of objects and not the image.