How to perform an image search with index lookup tool

Question

I have been trying to implement an image similarity search in PromptFlow. I created a vector index using an excel (.xlsx) file, with the following columns: filename (image filename), description (textual description of the image), embedding (embedding of the description). I then implemented an index lookup node in my flow, which does work. However, I now want to extract the filename out from the lookup, so that I can open the relevant file and output it to the flow. How can I achieve this?

Answer 1

Hi Seah Bryan,

Thanks for reaching out to Microsoft Q&A.

To extract the filename from the index lookup result in PromptFlow, you can modify your workflow as follows:

1. Ensure Proper Field Mapping: In your vector index, make sure that the column containing the filenames (say, 'filename') is included in the fields to be returned in the query result. This should be done when creating or updating the index so that Azure AI Search will return the filenames along with other fields during lookup.
2. Query and Access the Filename: When performing the index lookup, the search result will return the fields you specify. Make sure you retrieve the 'filename' field. For example, if you're using a search client, you would write the query such that it returns the 'filename' column. In your PromptFlow flow, after running the lookup node, you can access the result fields. Here's a general approach to accessing the 'filename' field from the result:

      # Assuming 'lookup_result' is the output from the index lookup node
      filename = lookup_result.get('filename', None)  # Retrieve filename
      # Assuming 'lookup_result' is the output from the index lookup node
      filename = lookup_result.get('filename', None)  # Retrieve filename
      
      
   

3. Open and Output the File: Once you have the filename, you can use it to fetch the corresponding file (assuming the file is stored in a location you have access to). For instance, if the file is in Azure Blob Storage or a similar service, you can open it using the appropriate SDK (like azure-storage-blob for azure blob storage). For instance:

      from azure.storage.blob import BlobServiceClient
      blob_service_client = BlobServiceClient.from_connection_string('your_connection_string')
      container_client = blob_service_client.get_container_client('your_container')
      blob_client = container_client.get_blob_client(filename)
      file_content = blob_client.download_blob().readall()
      # Output the file content to the flow (e.g., pass it to the next node)
      
      
   

4. Integrate with the Flow: Finally, ensure that you pass the retrieved filename and file content to subsequent steps in your PromptFlow process, depending on what you need to do with the file (ex: display, further process, etc.).

By ensuring that your query returns the relevant 'filename' field and then accessing it programmatically, you can fetch and manipulate the corresponding image file in your PromptFlow flow. Try and let me know.

Please 'Upvote'(Thumbs-up) and 'Accept' as an answer if the reply was helpful. This will benefit other community members who face the same issue.