SimilarProduct Insight not working in visual search API

Martin Meyer 0

i have this trying to find similiar products for an image of an product. Following the documentation
i just need put "SimilarProducts" in the invodedSkills of the api call
but i not get any answer with this code

endpoint = "https://api.bing.microsoft.com/v7.0/images/visualsearch"

image_url = "https://m.media-amazon.com/images/I/71CBk3JtF6L.__AC_SY300_SX300_QL70_ML2_.jpg"

# Specify the image insights token
knowledge_request = {
    "imageInfo": {
        "url": image_url  # Provide the URL of the image you want insights for
        #"imageInsightsToken": "ccid_cixTCTqz*cp_D7F0A068A89FA15BECFF781E1353DC4F"

    },
    "knowledgeRequest" : {


    "invokedSkills": ["SimilarProducts"],
    
}

    }


# Set the unique boundary
boundary = "boundary_1234abcd"

# Prepare the headers with multipart form-data
headers = {
    "Ocp-Apim-Subscription-Key": subscription_key,
    "Content-Type": f"multipart/form-data; boundary={boundary}"
}

# Format the multipart form-data body
body = (
    f"--{boundary}\r\n"
    f"Content-Disposition: form-data; name=\"knowledgeRequest\"\r\n\r\n"
    f"{json.dumps(knowledge_request)}\r\n"
    f"--{boundary}--"
)

# Send the POST request
response = requests.post(endpoint, headers=headers, data=body)

i get this response for an amazon product image

{"_type": "ImageKnowledge", "instrumentation": {"_type": "ResponseInstrumentation"}, "tags": [{"displayName": "", "actions": []}], "image": {"imageInsightsToken": "ccid_cixTCTqz*cp_D7F0A068A89FA15BECFF781E1353DC4F"}, "debugInfo": {}}

navba-MSFT 26,115 Microsoft Employee

@Martin Meyer Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

.
I tried to paste your image url in below and it worked fine:

https://www.bing.com/visualsearch

User's image

Could you please use the "invokedSkills": ["SimilarImages"] and test ?

I am sharing my working sample code here if that helps ?

import json
import requests
from pprint import pprint

'''
This sample uses the Bing Visual Search API to find similar images based on a provided image URL.
'''

# Add your Bing Search V7 subscriptionKey and endpoint.
endpoint = "https://api.bing.microsoft.com/v7.0/images/visualsearch"
subscription_key = "94d4XXXXXXXX7b142"

# Specify the image URL
image_url = "https://m.media-amazon.com/images/I/71CBk3JtF6L.__AC_SY300_SX300_QL70_ML2_.jpg"

# Prepare the headers
headers = {
    'Ocp-Apim-Subscription-Key': subscription_key,
    'Content-Type': 'multipart/form-data'
}

# Prepare the knowledge request
knowledge_request = {
    "knowledgeRequest": {
        "invokedSkills": ["SimilarImages"]
    }
}

# Prepare the file upload
files = {
    'image': ('image.jpg', requests.get(image_url).content)  # Download the image content
}

# Call the API
try:
    response = requests.post(endpoint, headers=headers, files=files, data={'knowledgeRequest': json.dumps(knowledge_request)})
    response.raise_for_status()

    print("\nHeaders:\n")
    print(response.headers)

    print("\nJSON Response:\n")
    pprint(response.json())
except Exception as ex:
    print(f"An error occurred: {ex}")

Hope this helps.

Martin Meyer 0 Reputation points

2024-11-07T07:48:05.35+00:00
@navba-MSFT Thanks for the answer!

the SimilarImages is working for me, but I am especially looking to also get the response object "aggregateOffer" with price and rating for each similar images.

Is there any other way to get the response object "aggregateOffer" https://learn.microsoft.com/en-us/bing/search-apis/bing-visual-search/reference/response-objects ?

The invokedSkill "SimilarProducts" is mentioned here in the docu. https://learn.microsoft.com/en-us/bing/search-apis/bing-visual-search/how-to/get-insights

The ImageInfo object includes the following fields:

url — The URL to the Internet-accessible image that you want insights of. The maximum supported image size is 1 MB. Read more.

imageInsightsToken — The insights token that Image Search API returned for an image. Read more.

cropArea — The bounding box that identifies the area of interest in the image. Read more.

The url and imageInsightsToken fields are mutually exclusive. If the request uploads the image binary in the body of the request, don’t includes the url and imageInsightsToken fields.

The KnowledgeRequest object includes the following fields:

filters — A Filter object that you use to filter the results.

site — The site filter returns similar images and similar products results from the specified site only. Read more.

invokedSkillsRequestData — A SkillsRequestData object that you use to request additional insights.

enableEntityData — A Boolean value that if true, returns information about the well-known person, place, or thing seen in the image. Read more.

invokedSkills — A list of insights to invoke. The string array may include only SimilarImages or SimilarProducts but not both. Include this field if you want to page visually similar insights or images of visually similar products. Read more. Specifying the image's information
navba-MSFT 26,115 Reputation points Microsoft Employee

2024-11-08T07:24:09.19+00:00

@Martin Meyer I am checking this internally. I will get back to you once I have an update on this.
Martin Meyer 0 Reputation points

2024-11-19T11:30:28.39+00:00

@navba-MSFT Did you manage to find any more information regarding this issue? If not, I may need to start evaluating other options.
navba-MSFT 26,115 Reputation points Microsoft Employee

2024-11-19T11:54:37.8433333+00:00

@Martin Meyer I heard back from the Product Owners. They are unable to reproduce this issue.
.

This is their response:

We’re seeing similar products alongside similar images, which is expected behavior. It's expected to see similar imager and products rather than only getting exact images and products.

.

Also, For example, If the user is shopping for a black cocktail dress and wants to see if the store they shop at carries it, use the site filter. The site filter restricts the similar images and similar products results to a specific domain.

Please refer for more details: - https://learn.microsoft.com/en-us/bing/search-apis/bing-visual-search/how-to/get-insights#filtering-similar-images-and-products
Martin Meyer 0 Reputation points

2024-11-19T13:56:00.34+00:00

@navba-MSFT Thanks for researching this !

Could you provide an example request body where it is working like described with similar images and products?
navba-MSFT 26,115 Reputation points Microsoft Employee

2024-11-20T04:33:00.63+00:00

@Martin Meyer Sure. I will share it once I get it from the Product owners.
navba-MSFT 26,115 Reputation points Microsoft Employee

2024-11-24T12:08:34.74+00:00

@Martin Meyer Attaching the Request headers used and the response headers received in txt file.

Request.txt
Response.txt
Martin Meyer 0 Reputation points

2024-11-27T17:04:05.0833333+00:00

@navba-MSFT Thanks for the files! In your example the response did not include price or rating of products found in the image. And i am still trying to replicate this example, but it is not working. https://learn.microsoft.com/en-us/bing/search-apis/bing-visual-search/how-to/get-insights#image-token-example My goal to get the shopping sources with price and rating for products found in an input image. Could you give me more support to archieve this ?
navba-MSFT 26,115 Reputation points Microsoft Employee

2024-11-28T04:20:28.44+00:00

@Martin Meyer Thanks for getting back. If you are facing challenges in achieving this, we may need a Support ticket to have a closer look at this issue. If you don't have a support plan to open a ticket, let me know. I can enable a one-time free support ticket for your subscription. Please let me know.
Martin Meyer 0 Reputation points

2024-11-28T09:36:50.76+00:00

@navba-MSFT I do not have a support plan, so i would really appreciate the option of having a one-time free support. Thanks in advance!

1 answer

navba-MSFT 26,115 Reputation points Microsoft Employee

2024-11-30T06:41:19.24+00:00

@Martin Meyer Thanks for sharing the details in private message. Due to the complexity of this issue, we may need a Support ticket to have a closer look at this issue.

.

After initial investigation on this issue, I have helped enable a one-time, courtesy free support flag on your subscription ‘428eXXXX-XXXX-XXXX-XXXX-XXXXacc7f7c3’ for a quick and immediate assistance.

Please create a service request to formally track this issue. This helps us track widespread impact in the event other customers are experiencing similar issues.

.

Details on how to raise a service request below:

•            Go to the Health Advisory section within the Azure Portal: https://aka.ms/healthadvisories

•            Select the Issue Name "You have been enabled for one-time Free Technical Support"

•            Details will populate below in the Summary Tab within the reading pane and you can click on the link "Create a Support Request" to the right of the message

.

Also once you have a resolution on this, please post it here so that it benefits other community audience.

.

**

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you.
Please sign in to rate this answer.

0 comments No comments
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

SimilarProduct Insight not working in visual search API

1 answer

Your answer