Build Your Own Image Similarity Model

  • Article

The Image Similarity with SQL Server solution on the Azure AI Gallery demonstrates how to apply transfer learning, incorporating a pretrained deep neural network (DNN) model (trained on ImageNet) to the problem of finding images that are similar to a target image. In this solution, we use an image based product recommendation scenario as an example, e.g., recommending clothing similar to the one a user is considering.

Our example features shirts (around 300 images) which contains 3 type of designs: dotted, striped, and leopard print. Those with similar patterns are considered more similar than those with different patterns.

Here are the steps we adopt to solve this problem:

  1. Use the pretrained model (ResNet-18) as a featurizer to transform the images into features
  2. Train a multi-class classifier on the features of training images. A 3-layer neural network from MicrosoftML package is used here.
  3. The Trained model is used to predict the class probabilities of each image.
  4. Calculate the distance (e.g., L1, L2, Cosine) between the predicted class probabilities of image pairs.
  5. Rank order image pairs based on the distance: the smaller the distance, the more similar of the image pairs.

This solution is developed to work on SQL Server ML Services available on SQL 2017. All the steps are written in Python and wrapped in SQL Server Stored procedures. It uses SQL Server FileTable to manage the storage of unstructured data, which is the type of setup a manufacturer of shirts might have for their inventory.

Are you ready to give it a try? On the deployed VM (or at the end of the on-prem installation), you'll find the following script running in a terminal window waiting for your reply:

 
This script will download all images listed in "c:\Solutions\ImageSimilarity\fashion_texture_urls.tsv"
onto your machine, upload them to SQL and execute the end-to-end workflow to train an image similarity model.
This portion of the script will take about 6 minutes to complete.

Please respond YES to continue

When you type YES, it will download the sample images, upload them to SQL Server and execute the end-to-end workflow of training and evaluating the image similarity model. You can simply replace the contents of the file with your own image URLs prior to responding to the prompt to get your own personalized model.

The hard part of the model building has become easy - now all you need is to go find the images you want to use to build your own image similarity model.

We’d love your feedback and looking forward to hearing what similar images you have been able to match!