Shelf Product Recognition (preview): Analyze shelf images using pretrained model
The fastest way to start using Product Recognition is to use the built-in pretrained AI models. With the Product Recognition API, you can upload a shelf image and get the locations of products and gaps.
Note
The brands shown in the images are not affiliated with Microsoft and do not indicate any form of endorsement of Microsoft or Microsoft products by the brand owners, or an endorsement of the brand owners or their products by Microsoft.
Prerequisites
- An Azure subscription - Create one for free
- Once you have your Azure subscription, create a Vision resource in the Azure portal. It must be deployed in a supported Azure region (see Region availability). After it deploys, select Go to resource.
- You'll need the key and endpoint from the resource you create to connect your application to the Azure AI Vision service. You'll paste your key and endpoint into the code below later in the guide.
- An Azure Storage resource with a blob storage container. Create one
- cURL installed. Or, you can use a different REST platform, like Swagger or the REST Client extension for VS Code.
- A shelf image. You can download our sample image or bring your own images. The maximum file size per image is 20 MB.
Analyze shelf images
To analyze a shelf image, do the following steps:
Upload the images you'd like to analyze to your blob storage container, and get the absolute URL.
Copy the following
curl
command into a text editor.curl -X PUT -H "Ocp-Apim-Subscription-Key: <subscriptionKey>" -H "Content-Type: application/json" "<endpoint>/computervision/productrecognition/ms-pretrained-product-detection/runs/<your_run_name>?api-version=2023-04-01-preview" -d "{ 'url':'<your_url_string>' }"
Make the following changes in the command where needed:
- Replace the
<subscriptionKey>
with your Vision resource key. - Replace the
<endpoint>
with your Vision resource endpoint. For example:https://YourResourceName.cognitiveservices.azure.com
. - Replace the
<your_run_name>
with your unique test run name for the task queue. It is an async API task queue name for you to be able retrieve the API response later. For example,.../runs/test1?api-version...
- Replace the
<your_url_string>
contents with the blob URL of the image
- Replace the
Open a command prompt window.
Paste your edited
curl
command from the text editor into the command prompt window, and then run the command.
Examine the response
A successful response is returned in JSON. The product recognition API results are returned in a ProductRecognitionResultApiModel
JSON field:
"ProductRecognitionResultApiModel": {
"description": "Results from the product understanding operation.",
"required": [
"gaps",
"imageMetadata",
"products"
],
"type": "object",
"properties": {
"imageMetadata": {
"$ref": "#/definitions/ImageMetadataApiModel"
},
"products": {
"description": "Products detected in the image.",
"type": "array",
"items": {
"$ref": "#/definitions/DetectedObject"
}
},
"gaps": {
"description": "Gaps detected in the image.",
"type": "array",
"items": {
"$ref": "#/definitions/DetectedObject"
}
}
}
}
See the following sections for definitions of each JSON field.
Product Recognition Result API model
Results from the product recognition operation.
Name | Type | Description | Required |
---|---|---|---|
imageMetadata |
ImageMetadataApiModel | The image metadata information such as height, width and format. | Yes |
products |
DetectedObject | Products detected in the image. | Yes |
gaps |
DetectedObject | Gaps detected in the image. | Yes |
Image Metadata API model
The image metadata information such as height, width and format.
Name | Type | Description | Required |
---|---|---|---|
width |
integer | The width of the image in pixels. | Yes |
height |
integer | The height of the image in pixels. | Yes |
Detected Object API model
Describes a detected object in an image.
Name | Type | Description | Required |
---|---|---|---|
id |
string | ID of the detected object. | No |
boundingBox |
BoundingBox | A bounding box for an area inside an image. | Yes |
tags |
TagsApiModel | Classification confidences of the detected object. | Yes |
Bounding Box API model
A bounding box for an area inside an image.
Name | Type | Description | Required |
---|---|---|---|
x |
integer | Left-coordinate of the top left point of the area, in pixels. | Yes |
y |
integer | Top-coordinate of the top left point of the area, in pixels. | Yes |
w |
integer | Width measured from the top-left point of the area, in pixels. | Yes |
h |
integer | Height measured from the top-left point of the area, in pixels. | Yes |
Image Tags API model
Describes the image classification confidence of a label.
Name | Type | Description | Required |
---|---|---|---|
confidence |
float | Confidence of the classification prediction. | Yes |
name |
string | Label of the classification prediction. | Yes |
Next steps
In this guide, you learned how to make a basic analysis call using the pretrained Product Recognition REST API. Next, learn how to use a custom Product Recognition model to better meet your business needs.