Shelf image composition (preview)

Άρθρο
09/10/2024

Important

This feature is now deprecated. On January 10, 2025, Azure AI Image Analysis 4.0 Custom Image Classification, Custom Object Detection, and Product Recognition preview API will be retired. After this date, API calls to these services will fail.

To maintain a smooth operation of your models, transition to Azure AI Custom Vision, which is now generally available. Custom Vision offers similar functionality to these retiring features.

Part of the Product Recognition workflow involves fixing and modifying the input images so the service can perform correctly.

This guide shows you how to use the Stitching API to combine multiple images of the same physical shelf: this gives you a composite image of the entire retail shelf, even if it's only viewed partially by multiple different cameras.

This guide also shows you how to use the Rectification API to correct for perspective distortion when you stitch together different images.

Prerequisites

An Azure subscription - Create one for free
Once you have your Azure subscription, create a Vision resource in the Azure portal. It must be deployed in a supported Azure region (see Region availability). After it deploys, select Go to resource.
- You'll need the key and endpoint from the resource you create to connect your application to the Azure AI Vision service. You'll paste your key and endpoint into the code below later in the quickstart.
An Azure Storage resource with a blob storage container. Create one
cURL installed. Or, you can use a different REST platform, like Swagger or the REST Client extension for VS Code.
A set of photos that show adjacent parts of the same shelf. A 50% overlap between images is recommended. You can download and use the sample "unstitched" images from GitHub.

Use the Stitching API

The Stitching API combines multiple images of the same physical shelf.

Three photos of shelves, next to their combined composite photo.

Note

The brands shown in the images are not affiliated with Microsoft and do not indicate any form of endorsement of Microsoft or Microsoft products by the brand owners, or an endorsement of the brand owners or their products by Microsoft.

To run the image stitching operation on a set of images, follow these steps:

Upload the images you'd like to stitch together to your blob storage container, and get the absolute URL for each image. You can stitch up to 10 images at once.

Copy the following curl command into a text editor.

curl.exe -H "Ocp-Apim-Subscription-Key: <subscriptionKey>" -H "Content-Type: application/json" "<endpoint>/computervision/imagecomposition:stitch?api-version=2023-04-01-preview" --output <your_filename> -d "{
    'images': [
        '<your_url_string_>',
        '<your_url_string_2>',
        ...
    ]
}"

Make the following changes in the command where needed:
1. Replace the value of <subscriptionKey> with your Vision resource key.
2. Replace the value of <endpoint> with your Vision resource endpoint. For example: https://YourResourceName.cognitiveservices.azure.com.
3. Replace the <your_url_string> contents with the blob URLs of the images. The images should be ordered left to right and top to bottom, according to the physical spaces they show.
4. Replace <your_filename> with the name and extension of the file where you'd like to get the result (for example, download.jpg).
Open a command prompt window.
Paste your edited curl command from the text editor into the command prompt window, and then run the command.

Examine the stitching response

The API returns a 200 response, and the new file is downloaded to the location you specified.

Use the Rectification API

After you complete the stitching operation, we recommend you do the rectification operation for optimal analysis results.

Photos of a retail shelf, before and after the rectify operation.

Note

To correct the perspective distortion in the composite image, follow these steps:

Upload the image you'd like to rectify to your blob storage container, and get the absolute URL.

Copy the following curl command into a text editor.

curl.exe -H "Ocp-Apim-Subscription-Key: <subscriptionKey>" -H "Content-Type: application/json" "<endpoint>/computervision/imagecomposition:rectify?api-version=2023-04-01-preview" --output <your_filename> -d "{
  'url': '<your_url_string>',
  'controlPoints': {
    'topLeft': {
      'x': 0.1,
      'y': 0.1
    },
    'topRight': {
      'x': 0.2,
      'y': 0.2
    },
    'bottomLeft': {
      'x': 0.3,
      'y': 0.3
    },
    'bottomRight': {
      'x': 0.4,
      'y': 0.4
    }
  }
}"

Make the following changes in the command where needed:
1. Replace the value of <subscriptionKey> with your Vision resource key.
2. Replace the value of <endpoint> with your Vision resource endpoint. For example: https://YourResourceName.cognitiveservices.azure.com.
3. Replace <your_url_string> with the blob storage URL of the image.
4. Replace the four control point coordinates in the request body. X is the horizontal coordinate and Y is vertical. The coordinates are normalized, so 0.5,0.5 indicates the center of the image, and 1,1 indicates the bottom right corner, for example. Set the coordinates to define the four corners of the shelf fixture as it appears in the image.
5. Replace <your_filename> with the name and extension of the file where you'd like to get the result (for example, download.jpg).
Open a command prompt window.
Paste your edited curl command from the text editor into the command prompt window, and then run the command.